The feature store in an automated MLOps architecture

Fully Automated MLOps

Arturo Opsetmoen Amador

Senior Consultant - Machine Learning

Features in machine learning

$$

Feature Engineering

Select, manipulate, and transform raw data sources to create features used as input for our ML algorithms

Picture of a worker wearing and suit next to paper sheets and cogs symbolizing a data process.

$$

Examples:

  • Numerical transformations
  • Encoding of categories
  • Grouping of values
  • Constructing new features
Fully Automated MLOps

Feature engineering in the enterprise

$$

Architecture figure showing data sources, batch and streaming, that are consumed by a team of data scientists transforming the data in a typical feature engineering process.

Fully Automated MLOps

Feature engineering in the enterprise

$$

The same architecture figure as previously depicted but with an additional team consuming the data sources working in their own silo with feature engineering.

Fully Automated MLOps

Feature engineering in the enterprise

$$

Continuation of the previous picture, yet a third team consuming the same data sources working with feature engineering in their own silo.

Fully Automated MLOps

The feature store

$$

  • Centralized feature repository

  • Avoid duplication of work with automation

  • Transformation standardization

  • Centralized storage

  • Feature serving for batch and real-time

$$

An architecture figure where the data sources are consumed into a feature store. Inside the feature store, data is transformed, the created features are stored, and finally, the feature store also serves the created features.

Fully Automated MLOps

The feature store - Accelerated experimentation

$$

  • Accelerated experimentation
    • Data extracts for experiments
    • Feature discovery
    • Avoids multiple definitions for identical features

$$

An architecture figure. The feature store lays horizontally and delivers features to orchestrated experiments in the Experimentation and Development environment.

Fully Automated MLOps

The feature store - Continuous training

$$

  • Continuous Training (CT)
    • Data extracts for automated pipelines in prod

$$

Architecture figure, The feature store lays horizontally. It delivers data to the continuous training component in the staging/production environments.

Fully Automated MLOps

The feature store - Online predictions

$$

  • Online predictions
    • Use pre-defined features for prediction services

$$

Architecture figure. The feature store lays horizontally. It delivers features to the prediction service in the system.

Fully Automated MLOps

The feature store - Environment symmetry

$$

  • Avoids training-serving skew

$$

An architecture figure. The feature store lays horizontally. It deliver features to orchestrated pipelines, continuous training and prediction services across environments.

Fully Automated MLOps

Let's practice!

Fully Automated MLOps

Preparing Video For Download...