Q&A
Flaky test quarantine in CI — auto-retry, dedicated quarantine job, or just fix them?
Ajitesh MohantaAmbassador
3w ago 1,678 0
Our flake rate is ~4% per test in CI (down from 9% after a big cleanup push). With 800 tests that means ~32 flaky failures per run on average.
Strategies I've seen:
- **Auto-retry**: `--retries=2` in Playwright. Works but masks the problem and slows CI.
- **Quarantine job**: failed-retry tests go into a separate "known flaky" pipeline that doesn't block PRs. Requires a tracking database.
- **Just fix them**: our current approach. Takes forever.
Is there a flaky test management tool that integrates with GitHub Actions that's actually worth using? I've looked at Buildpulse and Trunk Flaky Tests but haven't committed.