Guardrail Testing
Guardrail testing verifies that an AI system's safety mechanisms — content filters, topic restrictions, refusal behaviors — actually work as intended, correctly blocking harmful, off-topic, or out-of-scope requests without also incorrectly blocking legitimate ones.
Effective guardrail testing checks both failure directions, which is easy to test for only one of by accident: does the system correctly refuse genuinely harmful or out-of-scope requests (the guardrail working), and does it avoid over-refusing legitimate, benign requests that merely resemble something restricted on the surface (the guardrail being too aggressive).
It overlaps closely with prompt injection testing, since a common way to test guardrails is attempting known jailbreak and injection techniques specifically to see if the safety mechanism holds — a guardrail that only stops the most obvious, unsophisticated attempts isn't proven robust against a genuinely motivated adversary.
