Why you need to monitor your model

Monitoring Machine Learning Concepts

Hakim Elakhrass

Co-founder and CEO of NannyML

Machine learning in production

Typical development process

The arrow flow consists of four essential elements: defining the project, preparing the data, training the model, and deploying it.

After deployment

Reducing risk of failure

Zillow's case

A for sale board in front of a house.

Reasons for model to fail:

Software issues
Drifts in the input data
Changes in relationship between features and targets

Maximizing business impact

Optimizing the model in relation to business goals

Reducing cloud costs

The image displays a pyramid made up of six wooden blocks. The base layer involves creating a KPI from each letter on every block. The remaining layers represent growth, goal achievement, and increased efficiency.

The image displays the cloud logo, composed of nine wooden blocks.

Improving AI safety

Three safety problems:

Bias - fair output for different groups of users

Adversarial attacks - detect malicious manipulation of input data

Lack of explainability - understanding of how the model makes decisions

The image shows a wooden block with scale on it, being moved forward that symbolizes greater justice in the system, resulting in reduced bias.

The image shows a wooden block being moved forward with a keylock, which symbolizes increased security.

The image shows a wooden block being moved forward, with lines extending from the image of a head to represent an increase in understanding of the intelligence system.

Changing the world with data

The automatization process

The image shows a automated production line.

Let's practice!

Monitoring Machine Learning Concepts