MLOps on Kubernetes

Introduction to Kubernetes

Frank Heilmann

Platform Architect and Freelance Instructor

What is MLOps?

A paradigm to deploy and maintain machine learning models in production
A set of best-practice workflows with focus on continuous development of such models
Inspired by DevOps:
- Machine learning models are developed and tested in isolated experimental systems, and then deployed to production
- When in production, continuous monitoring; retraining may be triggered
Data scientists, data engineers, and IT teams can work on deployed models synchronously and ensure model accuracy

The MLOps paradigm maps very well to Kubernetes:
- Isolated experimental systems: can be realized via Pods and Kubernetes Storage
- Monitoring production ML models: enabled via lifecycle of Pods (and deployed image versions)
- Synchronous work on model accuracy: built in from the very beginning by Kubernetes architecture
Several frameworks for MLOps exist; the two best-known open-source solutions are
- mlflow
- Kubeflow

Kubeflow Overview

Kubeflow allows simple deployments of ML workflows specifically on Kubernetes
Covers each step of the ML model lifecycle
Consists of several components which cover these steps, working independently
Python can be used to develop and deploy ML models via an API
- no need to use kubectl

Introduction to Kubernetes