Experiment tracking

MLOps Concepts

Folkert Stijnman

ML Engineer

The machine learning experiment

machine learning experiment ingredients

MLOps Concepts

Why is experiment tracking important?

In each experiment, the following factors can be configured:

machine learning configurations

MLOps Concepts

Using experiment tracking in the ML lifecycle

Experiment tracking can help to:

  • Compare results
  • Reproduce past experiments
  • Collaborate with developers and stakeholders
  • Report on results to stakeholders
MLOps Concepts

How to track experiments?

Tool Pro Con
Spreadsheet Straightforward, easy to use Require a lot of manual work
Proprietary platform Custom solution specific for our process Require time and effort
Experiment tracking tool Specifically designed for experiments Requires getting familiar with the tool

 

MLOps Concepts

A machine learning experiment

Machine learning experiment examples

MLOps Concepts

The experiment process

  1. Formulate a hypothesis: "We expect that..."
  2. Gather images and labels
  3. Define experiments, e.g., types of models, hyperparameters, datasets
  4. Setup experiment tracking
  5. Train the machine learning model(s)
  6. Test the models on a hold-out test set
  7. Register the most suitable model
  8. Visualize and report back to team and stakeholders, and determine next steps

machine learning model training and testing

MLOps Concepts

Let's practice!

MLOps Concepts

Preparing Video For Download...