Feedback loop, re-training, and labeling

Machine Learning End-to-End

Joshua Stapleton

Machine Learning Engineer

Feedback loop

  • Model output considered as system input:
    • Using metrics/predictions to inform system evolution
    • Can use model monitoring

 

  • Integral part of ML:
    • Allows for rapid learning and adjustment
    • Better adapt to change

Two cyclical arrows showing the principle of a feedback loop

Machine Learning End-to-End

Feedback loop implementation

Data drift detection

  • Input data distribution changes over time
  • Feedback loop: retrain on newer data

Online learning

  • Periodically retrain based on changing data
  • Beyond data drift: adapts to changes in data structure

Data drift

Online Learning

Machine Learning End-to-End

Dangers of feedback loops

Dangers...

  • Model's outputs affect inputs
  • Eg: social media recommendation:
    • Maximize user engagement
    • Learns to serve certain type of content
    • Causes user to view more of this content
    • etc.
  • Develops undesired behavioral patterns
  • More dangerous when automated

A big red cross for danger

Machine Learning End-to-End

Better usage of feedback loop

  • Reactive:

    • Human in the loop
    • Model's predictions don't change input data

  • Caution and oversight are key!

A big green check for better usage

Machine Learning End-to-End

Let's practice!

Machine Learning End-to-End

Preparing Video For Download...