Communication management

Machine Learning for Business

Karolis Urbonas

Head of Machine Learning & Science, Amazon

Working groups

Schedule recurring meetings to track progress and define the following:

Define the business requirements
Review machine learning model and business products
Inference vs. prediction
Baseline model results & outline model updates
Market testing
Production

Business requirements

What is the business situation?
- Churn rate has started increasing
What is the business opportunity and how big is it?
- Reduce churn from X% to Y%
What are the business actions we will take?
- Run retention campaigns targeting customers at risk

Machine learning products

What ML products does the business needs?

$$

Example 1 - Predict churn. Business wants 1) inference into drivers of the churn updated quarterly, and 2) daily customer classification into: lost customers, customers at risk, no risk

$$

Example 2 - Fraud prediction. Business wants 1) inference into strong indicators of churn, and 2) real-time list of very risky transactions for manual review and medium risk ones for additional data request

Model performance and improvements

Identify what is the tolerance for model mistakes (remember - all models are wrong):

Classification
- Which class is more expensive to mis-classify?
- Example - it's likely more expensive to mis-classify fraud as non-fraud than vice versa
Regression
- What is the error tolerance for prediction?
- Example - in demand prediction the company will have to buy more inventory than needed if the model error is very high

Market testing

abtest

Machine learning in production

Are test results delivering consistent positive improvements?
Is the model stable enough?
Do we have systems and tools where the model be integrated to?

Let's practice!

Machine Learning for Business

Preparing Video For Download...