Synthetic UI fixtures for visual regression: a workflow that does not rot
1h ago 188 0
How do you keep visual regression suites from rotting when the fixture set is thin?
Our problem was not flakiness in the comparator, it was coverage. Cypress/Playwright screenshot tests were green for months while real regressions slipped through, because the same six pages were the only fixtures anyone had built. Fake accounts with three rows of data do not exercise truncation, empty states, long unbroken strings, or the odd CJK/RTL mixed content that breaks layouts in production.
What worked for us, in order:
1. Inventory the failure classes first, not the screens. Truncation, empty state, extremely long value, mixed script, dark-mode contrast, and 320px width covered about 80% of the visual bugs we had historically shipped.
2. Generate fixtures as data, not as screenshots. One seed per failure class, committed as JSON, loaded through the same factory the app uses. Reviewers can read the diff instead of eyeballing a PNG.
3. Only reach for image generation when the fixture has to *look* like something. Avatars, product thumbnails and hero images are the honest use case, because a solid grey box hides exactly the layout problems you are hunting for (aspect ratio drift, text over image, focal point cropping).
4. Pin the generated assets. Commit the files, hash them, and never regenerate during a test run. Regenerating mid-suite turns a visual diff into a random number generator.
For step 3 I have been using Nano Banana 2 Lite https://nanobanana2lite.tools/ as one option among several - it takes a prompt plus a reference object, which is handy when you need twenty thumbnails of the same imaginary product at different crops rather than twenty unrelated images. It is an independent third-party tool, not affiliated with Google or DeepMind, and the honest limitation is that it is a drafting aid: exact brand colours and legally required text still need a human pass.
Two things I would do differently: do not generate fixtures for anything that needs pixel-exact brand assets (you will spend more time matching them than drawing them), and never let generated images into a snapshot that runs in CI without a hash check.
Curious how others handle the fixture-library problem - especially teams where the design system ships faster than the test suite can follow.