The automation, monitoring, incident response pattern

Fully Automated MLOps

Arturo Opsetmoen Amador

Senior Consultant - Machine Learning

What is a software design pattern?

A general, reusable solution to a commonly occurring problem...

The image of a blueprint.

Automate, monitor, respond

An image showing a process: Automation, Monitoring, Incident Response.

Improves the reliability of the ML systems we design

Three examples of a design pattern in MLOps

Automated Model Retraining
Model Rollback
Feature Imputation

The image of a blueprint.

1. Automated model retraining

High level overview of the fully automated MLOps architecture.

1. Automated model retraining - running predictions

High level overview of the fully automated MLOps architecture. The prediction service is highlighted.

1. Automated model retraining - Monitoring

High level overview of the fully automated MLOps architecture. The performance monitoring module is highlighted.

1. Automated model retraining - Trigger

High level overview of the fully automated MLOps architecture. The trigger component is highlighted.

1. Automated model retraining - Automated pipeline

High level overview of the fully automated MLOps architecture. The automated pipeline is highlighted.

1. Automated model retraining - Deployment

High level overview of the fully automated MLOps architecture. The continuous delivery component is highlighted.

2. Model rollback

High level overview of the fully automated MLOps architecture. The automated pipeline is highlighted.

2. Model rollback - Validation fail

High level overview of the fully automated MLOps architecture. The model evaluation and model validation steps are highlighted.

2. Model rollback - Last functional model

High level overview of the fully automated MLOps architecture. The model registry is highlighted.

2. Model rollback - Redeployment

High level overview of the fully automated MLOps architecture. The continuous deployment component is highlighted.

3. Feature imputation - Data intensive pipeline

High level overview of the fully automated MLOps architecture. The data components in the automated pipeline are highlighted.

3. Feature imputation - Data quality

An image of a table showing data columns with missing values.

Varying levels of data quality
Some features might fall below a QA threshold

3. Feature imputation - Defective features

The columns in the previous table are highlighted and marked as defective.

Detect failing features
Apply feature imputation

3. Feature imputation - Potential fixes

Numerical Values

$$
- Mean/Median Imputation
- KNN Imputation

Categorical Values

$$
Frequent Category Imputation
Adding a "Missing" Category

Let's practice!