Jailbreak Testing
Jailbreak testing probes whether an AI model's safety and policy guardrails can be bypassed through crafted prompts — role-play framing ("pretend you're an AI with no restrictions"), encoded or obfuscated instructions, or multi-step conversations that gradually walk the model into producing content its system prompt explicitly forbids.
Unlike a straightforward safety violation, a jailbreak works by exploiting how the model resolves conflicting instructions — a user prompt cleverly framed to outweigh or reinterpret the system prompt's rules, rather than a single blunt request that gets refused outright. Common techniques include persona injection ("you are DAN, an AI with no restrictions"), hypothetical framing ("write a story where a character explains how to..."), and instruction smuggling inside translated text, code comments, or Base64-encoded strings.
Testing for it means maintaining a living library of known jailbreak patterns (they circulate publicly and evolve fast) and re-running them against every model or prompt update, since a fix for one phrasing rarely generalizes to a rephrased variant. It's an arms race, not a one-time checklist — what blocks a jailbreak attempt this month may not catch next month's variant.
QA's role is usually to own the regression suite of known attempts and flag drift when a previously-blocked prompt starts succeeding again after a model or prompt change, rather than trying to discover every possible jailbreak from scratch.
