Introduction to large language models (LLMs)

Introduction to LLMs in Python

Jasmin Ludolf

Senior Data Science Content Developer, DataCamp

Previous knowledge

  • Navigating the Hugging Face Hub
  • Deep learning models

Cartoon brain reading a book

Introduction to LLMs in Python

Introduction to LLMs

 

Icon of a lightbulb and brain to represent understanding

Introduction to LLMs in Python

Introduction to LLMs

 

Illustration of laptop to represent using

Introduction to LLMs in Python

Abstract illustration of a large language model

Introduction to LLMs in Python

Abstract illustration of a large language model with a summarize bubble

Introduction to LLMs in Python

Abstract illustration of a large language model with a summarize and generate bubble

Introduction to LLMs in Python

Abstract illustration of a large language model with a summarize, generate, and translate bubble

Introduction to LLMs in Python

Abstract illustration of a large language model with a summarize, generate, translate, Q&A bubble

LLM logos

Introduction to LLMs in Python

LLMs

 

  • Based on deep learning architectures

 

  • Most commonly transformers

 

  • Huge neural networks with lots of parameters and text data

Hugging Face logo

Introduction to LLMs in Python

Using Hugging Face models

from transformers import pipeline


summarizer = pipeline(task="summarization", model="facebook/bart-large-cnn")


text = "Walking amid Gion's Machiya wooden houses is a mesmerizing experience. The 
beautifully preserved structures exuded an old-world charm that transports visitors
back in time, making them feel like they had stepped into a living museum. The glow of 
lanterns lining the narrow streets add to the enchanting ambiance, making each stroll a
memorable journey through Japan's rich cultural history."

summary = summarizer(text, max_length=50)
  • clean_up_tokenization_spaces=True: remove unnecessary white space
Introduction to LLMs in Python

Model outputs

print(summary)

[{'summary_text': "Gion's Machiya
  wooden houses exuded an old-world 
  charm that transports visitors back 
  in time. The glow of lanterns lining 
  the narrow streets add to the 
  enchanting ambiance, making each 
  stroll a memorable journey through 
  Japan's"}]

print(summary[0]["summary_text"])

Gion's Machiya wooden houses exuded an 
old-world charm that transports 
visitors back in time. The glow of 
lanterns lining the narrow streets add 
to the enchanting ambiance, making each 
stroll a memorable journey through 
Japan's
Introduction to LLMs in Python

Up next

 

  • Build on existing LLM knowledge
  • Perform new tasks
  • See how LLMs are built
  • Fine-tune LLMs
  • Evaluate LLM performance

Learning path

Introduction to LLMs in Python

Let's practice!

Introduction to LLMs in Python

Preparing Video For Download...