Architectural components in end-to-end machine learning frameworks

End-to-End Machine Learning

Joshua Stapleton

Machine Learning Engineer

Feature stores

Features

Feature selection
Feature engineering

Feature store

Central repository for features
Ensures consistency, reduces duplication
Enables sharing, discovery
Standardizes feature transformations and calculations

Machine learning lifecycle image with feature store added

Feast

Popular tool for implementation of feature stores
Provides unified management, storage, serving, and discovery for ML features

Principles

Define, register features with feature sets
Feature sets: grouping of related features + metadata

Example: heart disease features

Patient entity
Associated features (cholesterol, age, sex)

Feast feature stores part 1

from feast import Field, Entity, ValueType, FeatureStore
from feast.data_source import FileSource


# Define the entity, which in this case is a patient, and features
patient = Entity(name="patient", join_keys=["patient_id"])

chol = Field(name="chol", dtype=Float32)
age = Field(name="age", dtype=Int32)
...

# Define the data source
data_source = FileSource(
    path="/path_to_heart_disease_dataset.csv",
    event_timestamp_column="event_timestamp",
    created_timestamp_column="created")

Feast feature stores part 2

# ... continued
# Create a feature view of the data
heart_disease_fv = FeatureView(name="heart_disease", entities=[patient],
    schema=[cholesterol, ...], ttl=timedelta(days=1), input=data_source,)


# Create a FeatureStore object
store = FeatureStore(repo_path=".")


# Register the FeatureView
store.apply([patient, heart_disease_fv])

Model registries

Model registry

Version control systems
Keep track of different versions of model
Annotate models
Track performance over time

Benefits

Organization
Transparency
Reproducibility

Machine learning lifecycle image with model registry added

Let's practice!

End-to-End Machine Learning