The automation, monitoring, incident response pattern

Fully Automated MLOps

Arturo Opsetmoen Amador

Senior Consultant - Machine Learning

What is a software design pattern?

$$

A general, reusable solution to a commonly occurring problem...

$$

$$

The image of a blueprint.

Fully Automated MLOps

Automate, monitor, respond

$$

$$

An image showing a process: Automation, Monitoring, Incident Response.

  • Improves the reliability of the ML systems we design
Fully Automated MLOps

Three examples of a design pattern in MLOps

$$

  1. Automated Model Retraining
  2. Model Rollback
  3. Feature Imputation

$$

The image of a blueprint.

Fully Automated MLOps

1. Automated model retraining

High level overview of the fully automated MLOps architecture.

Fully Automated MLOps

1. Automated model retraining - running predictions

High level overview of the fully automated MLOps architecture. The prediction service is highlighted.

Fully Automated MLOps

1. Automated model retraining - Monitoring

High level overview of the fully automated MLOps architecture. The performance monitoring module is highlighted.

Fully Automated MLOps

1. Automated model retraining - Trigger

High level overview of the fully automated MLOps architecture. The trigger component is highlighted.

Fully Automated MLOps

1. Automated model retraining - Automated pipeline

High level overview of the fully automated MLOps architecture. The automated pipeline is highlighted.

Fully Automated MLOps

1. Automated model retraining - Deployment

High level overview of the fully automated MLOps architecture. The continuous delivery component is highlighted.

Fully Automated MLOps

2. Model rollback

High level overview of the fully automated MLOps architecture. The automated pipeline is highlighted.

Fully Automated MLOps

2. Model rollback - Validation fail

High level overview of the fully automated MLOps architecture. The model evaluation and model validation steps are highlighted.

Fully Automated MLOps

2. Model rollback - Last functional model

High level overview of the fully automated MLOps architecture. The model registry is highlighted.

Fully Automated MLOps

2. Model rollback - Redeployment

High level overview of the fully automated MLOps architecture. The continuous deployment component is highlighted.

Fully Automated MLOps

3. Feature imputation - Data intensive pipeline

High level overview of the fully automated MLOps architecture. The data components in the automated pipeline are highlighted.

Fully Automated MLOps

3. Feature imputation - Data quality

$$

An image of a table showing data columns with missing values.

$$

  • Varying levels of data quality
  • Some features might fall below a QA threshold
Fully Automated MLOps

3. Feature imputation - Defective features

$$

The columns in the previous table are highlighted and marked as defective.

$$

  • Detect failing features
  • Apply feature imputation
Fully Automated MLOps

3. Feature imputation - Potential fixes

  • Numerical Values

    $$

    • Mean/Median Imputation

    • KNN Imputation

$$

  • Categorical Values

    $$

  • Frequent Category Imputation

  • Adding a "Missing" Category

Fully Automated MLOps

Let's practice!

Fully Automated MLOps

Preparing Video For Download...