Verifying generated images: read one thing, trace one line
4h ago 131 0
Most of my visual checks on generated output used to end at “does it look right?” That question is answered by the full frame, and it is a generous judge: a swapped object, a re-rendered background and a slightly different contact shadow all read as fine at page size.
The gap shows up when the edit was supposed to preserve something specific. I keep a test fixture with a small printed card in the corner — the identifer has to still read the same string after an edit that replaces the object next to it. At display size the card is a white rectangle. You cannot approve or reject that edit without a matching crop.
So the check I now run on any image edit, generated or otherwise: pick one thing to read and one line to trace, both outside the region that was supposed to change.
1) Read: the smallest text that must survive, at the size it will be displayed. I build the fixture in https://grokimage20.com/ — a plinth, a printed exhibit card, and two straight wall seams, so there is text to read and a geometric line to trace in the same frame.
2) Trace: the straightest edge near the edit boundary. If a wall seam bows by two pixels or the plinth rim shifts, the region that was meant to be untouched was not untouched.
The masking model matters here. Inpainting treats the mask as the permission boundary — pixels outside it are copied back from the source (https://huggingface.co/docs/diffusers/using-diffusers/inpaint). That guarantees the pipeline did not write outside the mask; it says nothing about whether what stayed inside still lines up with what surrounds it.
What this catches, in practice: identifiers that silently change (B-07 becoming B-O7 or 8-07), seams that drift, plinth edges that soften, contact shadows that detach from the object. What it does not prove: pixel-level fidelity of the untouched area. If the requirement is original pixels intact, composite instead and review the boundary where the two meet.
The cost is about ten seconds per edit and it converts a vibe check into a specific decision. Curious how others here handle visual regression on generated assets — do you keep a fixed fixture with readable text, or compare screenshot diffs against a baseline?