How businesses can become fully MLOps ready

MLOps for Business

Arne Jonas Warnke

Head of Emerging Curriculum

Where are we with regard to MLOps?

$$

Improving our MLOps capabilites requires

  • Assessing
  • Measuring

How can we do that?

$$

Using a magnifier to look at documents

MLOps for Business

MLOps maturity models

Microsoft:

  • 5 levels

relating to

  • Processes, technology, people

Google:

  • 3 levels

relating to

  • Level of automation

$$

Microsoft logo

$$

Google logo

MLOps for Business

Microsoft MLOps maturity model - level 0

$$

Level Description Highlights
0 No MLOps Siloed teams, manual processes
1 https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model
MLOps for Business

Microsoft MLOps maturity model - level 1

$$

Level Description Highlights
0 No MLOps Siloed teams, manual processes
1 DevOps no MLOps Siloed teams, automatic data gathering, first automatic tests
1 https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model
MLOps for Business

Microsoft MLOps maturity model - level 2

$$

Level Description Highlights
0 No MLOps Siloed teams, manual processes
1 DevOps no MLOps Siloed teams, automatic data gathering, first automatic tests
2 Automated Training Better collaboration, reproducibility, easier deployment
1 https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model
MLOps for Business

Microsoft MLOps maturity model - level 3

$$

Level Description Highlights
0 No MLOps Siloed teams, manual processes
1 DevOps no MLOps Siloed teams, automatic data gathering, first automatic tests
2 Automated Training Better collaboration, reproducibility, easier deployment
3 Automated Deployment Good collaboration, full reproducibility, traceability, automated testing
1 https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model
MLOps for Business

Microsoft MLOps maturity model - level 4

$$

Level Description Highlights
0 No MLOps Siloed teams, manual processes
1 DevOps no MLOps Siloed teams, automatic data gathering, first automatic tests
2 Automated Training Better collaboration, reproducibility, easier deployment
3 Automated Deployment Good collaboration, full reproducibility, traceability, automated testing
4 Automated Re-training Full collaboration, mastering operations, very few downtimes
1 https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model
MLOps for Business

Measuring our progress towards MLOps

$$

A maturity model is great

  • But how continuously measure our progress?
  • An MLOps KPI?

An image of a KPI process

MLOps for Business

Possible MLOps KPIs

$$

  • Deployment frequency
    • How many releases do we deploy per week?
  • Time to respond to incidences
    • If an error occurs, how much time do we need to restore our application?
  • Lead time for change requests
    • E.g., time to add a new feature.
1 https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance
MLOps for Business

Let's practice!

MLOps for Business

Preparing Video For Download...