Why one point at a time is not enough
Picture a boundary as a dot-to-dot drawing: a chain of points, each one free to slide back and forth along its own short line perpendicular to the edge, hunting for the spot where the pixel underneath it looks exactly half-covered. That is roughly what the earlier refinement stages do, and it works well for an isolated, well-separated edge.
It breaks down in two ways that a single point, working alone, cannot see. First, a pixel's colour is not only about the one boundary running near it — if several shapes meet close together, that pixel's colour is a blend of all of them, and no single point can account for that on its own. Second, a point can happily slide sideways, along the boundary rather than across it, without changing how any pixel looks at all — nothing in a one-point-at-a-time process stops it drifting, because no single pixel measurement objects.
So this stage treats the whole boundary — every point on it — as one set of unknowns to be solved together, against one shared yardstick: how closely the shape, if rendered honestly, would reproduce the actual picture.
The yardstick: paint it and compare
The honest way to check whether a boundary is in the right place is to ask what a careful renderer would do with it — not "is the point near the edge", but "if I clipped this exact pixel square by this exact boundary, what fraction of it falls on each side, and does mixing the two colours in that proportion match what the pixel actually shows?"
That clipped fraction can be computed exactly, as a plain area calculation, and — this is what makes solving thousands of points at once practical rather than impossibly slow — the way that area changes as any one point moves can also be worked out exactly, by algebra, instead of by nudging the point slightly and re-measuring. Nudging-and-re-measuring for every point against every pixel would be far too slow; having the exact formula in hand is what lets the whole boundary move together in reasonable time.
Two habits of mind, built in
Matching pixels exactly is not, by itself, enough to produce a sensible boundary — an infinite number of wiggly shapes can match a set of pixels equally well (more on that below). Two additional preferences are built into the maths, both scaled automatically to the size and contrast of the particular image so they mean the same thing on a plain two-colour logo and on a crowded, busy emoji:
- Prefer a smooth line to a corner, and a gentle corner to a sharp one. Corners are allowed — a sharp corner in the artwork should stay sharp — but a corner costs something in proportion to how sharp it is, so the solver does not spend one big, honest corner's worth of "cost" on ten small, meaningless zigzags instead.
- Do not drift without reason. Every point starts from where the earlier, careful measurement put it, and unless the picture gives it good reason to move, it stays close to that starting position — especially along the boundary's own length, in the direction a pixel-matching test cannot see at all.
Where three shapes meet, trust the intersection, not one pixel
Where two boundaries meet a third — a corner where three colours come together — a single pixel there is a blend of three or more shapes at once, and reading a clean signal out of that blend is much less reliable than reading it from an ordinary two-colour edge. This stage handles that by holding those meeting points firmly in place, several times more firmly than an ordinary point, and by default leaves them out of the pixel-matching comparison entirely: they still move, because the ordinary boundary points on either side of them pull them along, but the pixels right at the meeting point do not get a vote on exactly where it sits. An earlier stage has already placed that point by working out precisely where the two boundary lines geometrically cross, which is sturdier evidence than one blended pixel.
This was tried the other way — letting those meeting-point pixels vote too — and it made results measurably worse across a large test set of icons, not better. Junctions in real artwork are rare enough, and the blended-pixel signal there weak enough, that trusting them costs more than it buys.
The sawtooth: a real limit of the idea, honestly reported
Matching pixels exactly has a blind spot. Picture a thin ribbon — a stroke about two pixels wide. Its two long edges are close enough together that they are competing for the same pixels. If one edge nibbles a little area away from a pixel and the other edge, right next to it, gives a little area back, the total picture is unchanged — the maths cannot tell the difference between a smooth straight edge and a jagged, saw-toothed one that happens to trade area back and forth in exactly this way.
The team measured, and rejected, four different general-purpose fixes for this — turning up the smoothness preference further, adding a penalty for how long the boundary is, weighting each point by how confidently it was originally measured, and simply stopping the solver earlier before the teeth have time to grow. Every one of them either failed to remove the sawtooth, or removed it at the cost of blurring away real, wanted detail somewhere else in the corpus of test images. None of the four was shipped.
A safety catch: never let the fix tie the boundary in a knot
Because points move together rather than one at a time, there is a real risk that the solved answer accidentally pulls one side of a boundary through the other — imagine pulling both edges of that thin ribbon toward the same strip of dark pixels until they cross. A crossed boundary is not a small cosmetic problem downstream; it can make a shape's inside and outside ambiguous and force a lot of expensive repair work later.
So after solving, the stage checks whether it introduced any new crossing that was not already there before it started (an existing tangle from earlier stages is a separate problem, not this stage's to fix). If it did, it does not throw the whole improvement away — it scales the entire proposed motion back toward zero, a little at a time, until the new crossing disappears. Only if it cannot back off far enough to avoid it does it give up and leave the boundary exactly as it found it.
How long to keep solving
Like any iterative process, this one can be stopped early or run longer. Early testing used a smaller number of passes; a careful measurement later found that setting was stopping the process before it had actually settled down — the boundary handed to the next stage was still visibly moving. Running roughly twice as many passes let it finish properly, and measured across the project's full test set, every family of icons improved or held steady on accuracy and got cheaper to describe at the same time — which is the signature of a process that has genuinely converged, rather than one that has simply been tuned to trade one thing for another. The extra passes cost only a few milliseconds per image.
Interestingly, running for far more passes than that does not keep helping, and on one measurement set it very slightly hurt. The explanation on record is that once the true signal in the pixels has been mostly extracted, additional passes are chasing smaller and smaller effects where the process's own step-taking behaviour starts to wander rather than usefully improve the picture.