What is concept drift?

Monitoring Machine Learning Concepts

Hakim Elakhrass

Co-founder and CEO of NannyML

Definition

Change in relationship between the model inputs and the target

$P(Y|X)$ changes, $P(X)$ stays the same

The image shows change in the relationship between income and age in production comparing to the training environment.

Why drift happens?

External events
- viral trends, policy changes
Unmodeled seasonality
- in case of demand forecasting seasonal events like Black Friday or Christmas
Changes in data-generation process
- new update to the data collection app
Evolving user behavior
- habits, patterns, preferences are constantly changing

The dynamics of concept drift

Sudden drift - a new concept occurs within the short time

Gradual drift - a new concept gradually replaces the old one

Reoccurring - reoccurring old concept over time

Image shows text Covid-19.

Image shows American banknotes.

Image shows poster with 50% of sales during Black Friday.

Effects of covariate shift on concept drift

Negative
- the effect of concept drift decreases

Positive
- the effect of concept drift intensifies

The image illustrates a covariate shift between the training and production phases. Specifically, the income feature has shifted from 80k/year to 100k/year. In the second graph, the image depicts concept drift, indicating a change in the relationship between income and the predicted loan default. Consequently, there is a higher probability of loan default for applicants with lower incomes.

The image illustrates a covariate shift between the training and production. Specifically, the income feature has shifted from 80k/year to 60k/year. In the second graph, the image depicts concept drift, indicating a change in the relationship between income and the predicted loan default. Consequently, there is a higher probability of loan default for applicants with lower incomes.

Let's practice!

Monitoring Machine Learning Concepts