PII Leakage Testing
PII leakage testing checks whether an AI model exposes personally identifiable information it shouldn't — training data it memorized, another user's data from a shared context window, or details it can be prompted into revealing — by deliberately probing it with prompts designed to extract sensitive information.
LLMs can memorize fragments of their training data verbatim, and in a multi-tenant application, a poorly isolated context — shared chat history, a cache keyed incorrectly — can leak one user's data into another user's session. Both are PII exposure risks that a purely functional test suite has no reason to check for, since the feature "works" in the ordinary sense either way.
Testing usually combines two approaches: extraction probing, using crafted prompts that try to get the model to repeat memorized training data, and isolation testing, verifying that session and context boundaries actually hold under concurrent or malformed requests, so one user's data genuinely never reaches another user's response.
This is especially critical for any AI feature built on retrieval-augmented generation over internal documents — a RAG system that retrieves from the wrong permission-scoped index can hand a user someone else's private data formatted as a confident, on-topic answer, which looks nothing like a typical access-control bug and is easy to miss without dedicated tests.
