Monitoring machine learning models
MLOps Concepts
Folkert Stijnman
ML Engineer
Monitoring & retraining
Monitoring
Types of monitoring
Statistical monitoring: focuses on the input and output data, including predictions
Examples: customer X has a 72% probability of churning, customer Y has a 31% probability of not churning
Types of monitoring
Computational monitoring: focuses on technical metrics
Examples: server CPU usage, number of incoming requests, number of predictions, downtime of server
Statistical and computational monitoring
Feedback loop
Feedback loop: the process through which the ground truth is used to improve the machine learning model
Monitoring in production
