Concept Drift Testing
Concept drift testing checks for cases where the relationship between input and correct output has changed over time — even if the input data itself looks statistically similar — so a model that was accurate when trained becomes systematically wrong simply because the world it's modeling has moved on.
Concept drift is easy to confuse with data drift but is a distinct failure: data drift is the inputs looking different, concept drift is the same kind of input now having a different correct answer. A fraud-detection model is a classic example — the input transactions might look statistically similar to training data, but fraud patterns evolve specifically to evade the last known detection method, so the mapping from "input" to "is this fraud" genuinely changes even when the inputs themselves haven't.
Detecting concept drift generally requires tracking actual ground-truth outcomes over time, not just input distributions, and watching for accuracy degradation on fresh, recently-labeled data even when input-distribution monitoring shows nothing unusual — which is exactly why relying on data drift detection alone gives a false sense of safety. This is one of the strongest arguments for maintaining a continuous pipeline of freshly labeled evaluation data rather than validating only against a static historical set.
