Model maintenance

Deployment e ciclo di vita in MLOps

Nemanja Radojkovic

Senior Machine Learning Engineer

Previously...

  • How to catch anomalies and disturbances?

 

Focus of this lesson: ML model failure

Deployment e ciclo di vita in MLOps

model deteriorated

Deployment e ciclo di vita in MLOps

improve the model

Deployment e ciclo di vita in MLOps

improve dataset

Deployment e ciclo di vita in MLOps

model centric

Deployment e ciclo di vita in MLOps

data centric

1 https://datacentricai.org/
Deployment e ciclo di vita in MLOps

Quality above quantity

  • Get more features with relevant information
  • Get better labels

 

Label == Target variable value in the training set

Label quality == label closeness to the ground truth

Deployment e ciclo di vita in MLOps

poor labels

Deployment e ciclo di vita in MLOps

poor prediction

Deployment e ciclo di vita in MLOps

Benefits of labeling tools

Manual labeling is complex, lengthy, error-prone

Use good labeling tools!

  • Labeling more efficient
  • More accurate
  • User interface fit for purpose
  • "Label these examples first for maximum impact"
  • "It seems you made a mistake here"

labeling tool

Example: Image labeling tool, for building image classifiers

Deployment e ciclo di vita in MLOps

human in the loop

Deployment e ciclo di vita in MLOps

HIL 2

Deployment e ciclo di vita in MLOps

expert input

Deployment e ciclo di vita in MLOps

Deployment e ciclo di vita in MLOps

When the labels arrive

  • Run ML build pipeline => New model
  • Check if new model is better than the old one
    • YES => test and deploy
    • NO => keep searching,
Deployment e ciclo di vita in MLOps

Keep searching

  • Try new models
  • Try new features
  • Try new data sources

 

Immensely helpful: Metadata store (MLFlow Tracking, etc)

  • Document the model selection journey
  • Avoid repeating same experiments

 

In any case: MLOps helps us maintain our model in the fastest, most efficient way

Deployment e ciclo di vita in MLOps

Let's practice!

Deployment e ciclo di vita in MLOps

Preparing Video For Download...