Introduction to Continuous Integration/Continuous Delivery for Machine Learning

CI/CD for Machine Learning

Ravi Bhadauria

Machine Learning Engineer

SDLC Overview

  • SDLC: Software Development Life Cycle
  • Systematic approach covering software development from start to finish
  • SDLC workflow refers to the sequence of steps followed to achieve specific goals:
    • Build: compiling and/or packaging code, resolving dependencies
    • Test: used to ensure codebase functionality, quality, and reliability
    • Deploy: process of making the software available for use in a specific environment

  Dependency plot of SDLC workflow steps. Build is first step, followed by test and deploy.

CI/CD for Machine Learning

SDLC in machine learning

  • Machine learning development can be complex and time-consuming
    • Model is an algorithm that evolves dynamically
    • Data engineering is important
  • Continuous Integration/Continuous Delivery reduces errors and ensures faster delivery of high-quality ML software
  • Essential for efficient machine learning and experimentation
1 https://cloud.google.com/blog/products/ai-machine-learning/making-the-machine-the-machine-learning-lifecycle
CI/CD for Machine Learning

What is CI/CD?

  • Continuous Integration (CI): The practice of frequently building, testing, and merging code changes into a shared repository
  • Allows developers to detect integration issues early and maintain a consistent codebase
  • Continuous Delivery (CD): Ensures that code changes can be deployed to production at any time but requires manual approval
  • Continuous Deployment (CD): Automatically deploys code changes to production without manual intervention
CI/CD for Machine Learning

CI/CD in machine learning

  • Data Dependency: Data versioning and management strategies
  • Experimentation: Automating hyperparameter tuning
  • Model Versioning: Improving collaboration
  • Testing Paradigm: Goes beyond traditional functional and unit testing
  • Continuous Deployment Challenges: Complexities in model serving, monitoring, and updates
CI/CD for Machine Learning

Scope of this course

Plot of machine learning project steps, emphasizing Data Preparation, Model Development, and Model Evaluation and Tuning steps.

CI/CD for Machine Learning

Summary

  • Software Development Life Cycle workflow involves building, testing, and deploying code
  • Continuous Integration (CI) ensures frequent code merging and early issue detection
  • Continuous Delivery (CD) allows code changes to be deployed with manual approval
  • Continuous Deployment (CD) automates code deployment without manual intervention
  • CI/CD in Machine Learning enables
    • Data versioning
    • Building models and model versioning
    • Automating experiments
    • Testing
    • Deployment
CI/CD for Machine Learning

Let's practice!

CI/CD for Machine Learning

Preparing Video For Download...