Overview of LLMOps

LLMOps Concepts

Max Knobbout, PhD

Applied Scientist, Uber

What we will learn in this course

A playful illustration of a book sitting at its desk, learning

 

  • LLMOps helps to effectively manage, deploy, and maintain large language model applications
  • We will cover:
    • Fundamentals
    • Lifecycle of LLM applications
    • Challenges and considerations
1 Illustrations by Manfred Steger @ Pixabay
LLMOps Concepts

A recap of LLMs

 

 

What are LLMs?
  • 📚 Trained on extensive text data
  • 🧠 Can understand and generate human-like text
  • 💡 Represent an AI breakthrough

 

 

What sets them apart?
  • 🏋 Typically pre-trained
  • 🐘 Massive number of parameters
  • 💻 Significant computational resources
  • 🎲 Unpredictable
LLMOps Concepts

How it started...

  • Queries were directly fed into the model
  • The focus was on operating the model
  • Only when the model was fine-tuned data was introduced

A simple pipeline depicting a query going into an LLM and generating an output

LLMOps Concepts

... versus how it's going

  • Integrating organizational data before text generation
  • Steps can involve data processing and manipulation
  • One, or multiple model calls, accommodating text, image, or multi-modal

Resulting in what we call LLM applications throughout this course

Picture of a complex LLM pipeline

LLMOps Concepts

The need for LLMOps

 

 

Seamless integration

LLMOps Concepts

The need for LLMOps

 

 

Smooth lifecycle transitions

LLMOps Concepts

The need for LLMOps

 

 

Efficient, scalable management

LLMOps Concepts

LLMOps versus MLOps

LLMOps Concepts

LLMOps versus MLOps

Some differences:

LLMOps MLOps
Model size Large Typically smaller
Data Text Any data
Pre-trained models Typically yes Typically no
Model improvement Prompt engineering & fine-tuning Feature engineering & model selection
Generalization General-purpose Fixed scope
Unpredictability High Low
Output Primarily text Task-specific
LLMOps Concepts

Let's practice!

LLMOps Concepts

Preparing Video For Download...