Experiment tracking

Concetti MLOps

Folkert Stijnman

ML Engineer

The machine learning experiment

machine learning experiment ingredients

Concetti MLOps

Why is experiment tracking important?

In each experiment, the following factors can be configured:

machine learning configurations

Concetti MLOps

Using experiment tracking in the ML lifecycle

Experiment tracking can help to:

  • Compare results
  • Reproduce past experiments
  • Collaborate with developers and stakeholders
  • Report on results to stakeholders
Concetti MLOps

How to track experiments?

Tool Pro Con
Spreadsheet Straightforward, easy to use Require a lot of manual work
Proprietary platform Custom solution specific for our process Require time and effort
Experiment tracking tool Specifically designed for experiments Requires getting familiar with the tool

 

Concetti MLOps

A machine learning experiment

Machine learning experiment examples

Concetti MLOps

The experiment process

  1. Formulate a hypothesis: "We expect that..."
  2. Gather images and labels
  3. Define experiments, e.g., types of models, hyperparameters, datasets
  4. Setup experiment tracking
  5. Train the machine learning model(s)
  6. Test the models on a hold-out test set
  7. Register the most suitable model
  8. Visualize and report back to team and stakeholders, and determine next steps

machine learning model training and testing

Concetti MLOps

Let's practice!

Concetti MLOps

Preparing Video For Download...