Ideation phase

LLMOps Concepts

Max Knobbout, PhD

Applied Scientist, Uber

LLM lifecyle: Ideation phase

Overview of the LLM application lifecycle phases

LLMOps Concepts

Data sourcing

 

Playful picture of a cartoon holding up different sources of data

 

  • Identifying needs

 

  • Finding sources

 

  • Ensuring accessibility
LLMOps Concepts

Data sourcing

 

Playful picture of a cartoon holding up different sources of data

  1. Is the data relevant?
  2. Is the data available?
    • Transform the data
    • Set up additional databases
    • Evaluate costs
    • Consider other access limitations
  3. Does the data meet standards?
    • Concerns quality and governance
LLMOps Concepts

Selecting the base model

  • Pre-trained models:

Picture of a pre-trained model

  • Proprietary or open-source?
LLMOps Concepts

Proprietary models (privately owned)

Advantages:

  • Ease of set-up and use
  • Quality assurance
  • Reliability, speed, and availability

 

Limitations:

  • Requires exposing data
  • Customization

Examples:

Logo's of proprietary LLM models

LLMOps Concepts

Open-source (publicly accessible)

Advantages:

  • In-house hosting
  • Transparency
  • Full customizability

 

Limitations:

  • Support
  • Commercial use

 

Example: Logo of Meta Llama 3

 

Downloadable from: Logo of Hugging Face

LLMOps Concepts

Factors in model selection

1. Performance ⭐

  • Response quality
  • Speed

Icons related to performance

2. Model Characteristics ⭐

  • Data used to train the model
  • Context window size
  • Fine-tunability

Icons related to model characteristics

LLMOps Concepts

Factors in model selection

3. Practical Considerations ⭐

  • License
  • Cost
  • Environmental impact

Icons related to practical considerations

4. Secondary factors

  • Number of parameters
  • Popularity

Icons related to secondary factors

LLMOps Concepts

Let's practice!

LLMOps Concepts

Preparing Video For Download...