Overview of LLMOps

Concetti di LLMOps

Max Knobbout, PhD

Applied Scientist, Uber

What we will learn in this course

A playful illustration of a book sitting at its desk, learning

 

  • LLMOps helps to effectively manage, deploy, and maintain large language model applications
  • We will cover:
    • Fundamentals
    • Lifecycle of LLM applications
    • Challenges and considerations
1 Illustrations by Manfred Steger @ Pixabay
Concetti di LLMOps

A recap of LLMs

 

 

What are LLMs?
  • 📚 Trained on extensive text data
  • 🧠 Can understand and generate human-like text
  • 💡 Represent an AI breakthrough

 

 

What sets them apart?
  • 🏋 Typically pre-trained
  • 🐘 Massive number of parameters
  • 💻 Significant computational resources
  • 🎲 Unpredictable
Concetti di LLMOps

How it started...

  • Queries were directly fed into the model
  • The focus was on operating the model
  • Only when the model was fine-tuned data was introduced

A simple pipeline depicting a query going into an LLM and generating an output

Concetti di LLMOps

... versus how it's going

  • Integrating organizational data before text generation
  • Steps can involve data processing and manipulation
  • One, or multiple model calls, accommodating text, image, or multi-modal

Resulting in what we call LLM applications throughout this course

Picture of a complex LLM pipeline

Concetti di LLMOps

The need for LLMOps

 

 

Seamless integration

Concetti di LLMOps

The need for LLMOps

 

 

Smooth lifecycle transitions

Concetti di LLMOps

The need for LLMOps

 

 

Efficient, scalable management

Concetti di LLMOps

LLMOps versus MLOps

Concetti di LLMOps

LLMOps versus MLOps

Some differences:

LLMOps MLOps
Model size Large Typically smaller
Data Text Any data
Pre-trained models Typically yes Typically no
Model improvement Prompt engineering & fine-tuning Feature engineering & model selection
Generalization General-purpose Fixed scope
Unpredictability High Low
Output Primarily text Task-specific
Concetti di LLMOps

Let's practice!

Concetti di LLMOps

Preparing Video For Download...