Algorithms (5 blogmarks)

In AI Learns to Optimize Factorio Blueprints, I learned about Wave Function Collapse. It was used to eliminate options of where to place certain tiles on a grid using neighbor-based constraints. This is a form of constraint propagation.

The linked blog post goes into this in much more detail using Sudoku to demonstrate it. It builds more foundation for the concepts including going into constraint programming (constraint solving algorithms).

Guessing and backtracking eventually terminates.

Every time you backtrack you reduce the domain of at least one variable, so even though there is a lot of giving up and restarting involved, the algorithm is guaranteed to eventually terminate.

This all relates to wave function collapse in that you want to settle on a series of tile placements that all satisfy one another’s’ constraints.

In WFC, the goal is to fill in a grid with tiles such that nearby tiles connect to each other. In the terminology we used above, each tile is a different value, and each cell in the grid is a variable representing the choice of tile. And the rules about which tiles can be placed where are the constraints.

A mini-game used to disguise an interactive learning resource on optimal path finding and The Traveling Salesman Problem.

The general theme here is "less is more". Don't write anticipatory code for potential bottlenecks, you need to measure and identify them first. Start with simpler algorithms. Start with simpler data structures.

Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is.

Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest.

Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy. (Even if n does get big, use Rule 2 first.)

Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder to implement. Use simple algorithms as well as simple data structures.

Rule 5. Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming.

This last bit of Rule 5 is particularly interesting: "Data structures, not algorithms, are central to programming."

It is so much easier to reason about and build a flow of logic when you've modeled the problem well and gotten the data structure right.

I enjoyed reading about this algorithm because it demonstrates some clever tricks and problem solving. It's useful to read about new algorithms from time to time if for no other reason than as a source of inspiration to keep you actively thinking about unique was to solve problems.

A high level summary of what makes it so neat: the algorithm uses adaptive bias adjustment, variable-length encoding, and delta compression to achieve extremely dense (in the information-theoretic sense) results for both single-script domains (like German bücher-café.de or Chinese 北京-旅游.cn) and mixed-script domains (like hello世界.com). What makes it particularly elegant is that it does this without any shared state between encoder and decoder. The adaptation is purely a deterministic function of the encoded data itself. This is a really cool example of how you can use the constraints of a system to your advantage to solve a problem.

Reservoir sampling is a technique for selecting a fair random sample when you don't know the size of the set you're sampling from.