Data Drift Monitoring для ML моделей

Data drift — изменение распределения данных после deployment. Причина деградации ML моделей в production.

Типы drift

Covariate shift: изменение distribution features (новые user demographics). Prior probability shift: изменение frequency labels (больше fraud attempts). Concept drift: изменение отношения features → target (fraud patterns evolve). Все требуют разной реакции.

Detection methods

Statistical tests: KS test, Chi-square для distributions. Population Stability Index (PSI). Kullback-Leibler divergence. Model-based: performance metrics degradation (AUC drop). Feature attribution changes. Prediction distribution shifts. Threshold-based alerts.

Инструменты

Evidently AI: open-source drift monitoring. WhyLabs: managed platform. Fiddler: enterprise. AWS SageMaker Model Monitor. Great Expectations для data quality. Custom pipelines через Prometheus. Integration с alerting.

Response

Investigation: root cause analysis. Retraining: обычно fix для gradual drift. Emergency model rollback при sudden drop. Feature engineering: adaptations для new patterns. A/B testing new vs old. Continuous automated retraining pipelines. Business impact assessment.