Model Drift Testing
Model drift testing detects when a deployed model's performance degrades over time — because real-world input has shifted away from the data it was trained on, or the underlying task itself has changed — catching gradual quality decay that a one-time pre-deployment evaluation wouldn't reveal.
A model evaluated thoroughly before deployment can still degrade afterward as the world it operates in changes — user behavior shifts, new slang or terminology emerges, or the underlying data distribution simply moves away from what the model originally learned — none of which shows up in a single pre-launch evaluation.
Detecting drift typically means continuously monitoring live performance metrics (not just running the fixed golden dataset once) and periodically re-evaluating against fresh, recently collected real-world examples — treating model quality as an ongoing production concern, similar in spirit to synthetic monitoring for traditional software.
