Experimentation

Concevoir des pipelines de prévision pour la production

Rami Krispin

Senior Manager, Data Science and Engineering

Experimentation

Forecasting Pipeline

Experimentation in a nutshell

Problem statement

Experimentation in a nutshell

Setting the hypothesis

Experimentation in a nutshell

Hypothesis testing

Experimentation in a nutshell

Repeating the process for additional hypotheses

Experimentation in a nutshell

Reaching a conclusion

Experimentation in forecasting

Experimentation identifies the best (forecasting) modeling approach for each question.

Data
Models
Training framework
Performance KPIs
Model registration process
Model selection

A typical workflow covering data > modeling > model registration > model selection

Experimentation in forecasting

DataFrame containing several forecasting models and their associated metrics - MAPE, RMSE, and Coverage

Experimentation in forecasting

DataFrame with the AutoARIMA model highlighted

AutoARIMA documentation displaying several arguments that can be used when instantiating the model

Experimentation in forecasting

DataFrame with the MSTL_ARIMA_trend model highlighted

MSTL_ARIMA_trend documentation highlighting that a different trend forecaster can be set

Workflow

Workflow of train > test > evaluate > deploy > monitor > re-tune > repeat

Workflow

Architecture consisting of data > data processing > backtesting > scoring > logging and metadata

Workflow

Data processing

Workflow

Backtesting

Workflow

Scoring

Workflow

Logging

Workflow

Model settings defined in a JSON file

Workflow

Technologies used in the architecture - pandas for data processing, nixtla for backtesting and scoring, and MLflow for logging

Let's practice!

Concevoir des pipelines de prévision pour la production

Preparing Video For Download...