GPT models and training LLMs

Intermediate ChatGPT

Alex Banks

Founder & Educator

The genesis of GPT

The image features a surprised man on the left, looking at three purple bars of increasing size labeled GPT-1, GPT-2, and GPT-3 from left to right, representing the growth in size of each version

The genesis of GPT

An illustration shows a man standing looking toward an oval shape, representing a neuron

Breaking down GPT

An illustration shows a confused man with a question mark above his head. To his right, the text 'Generative Pre-trained Transformer' is displayed, with an arrow pointing to 'GPT' in bold and underlined

An illustration shows a man writing in a notebook with a pencil. To his left, the word 'Generative' is displayed

Illustration of a man running with books and electronic devices trailing behind him, with the word 'Pre-trained' to the right

Illustration of a man transforming from one state to another, with the word 'Transformer' above them

Illustration of a man sitting with a laptop, next to the ChatGPT logo

How do we train LLMs?

Illustration of a woman reading a book standing next to a large robot, both looking at the book

Step 1 - Pre-training

Large quantities of low-quality internet data Illustration of a woman holding a clipboard, with a chart and a table detailing the pre-training process and data of GPT models

Step 2 - Fine-tuning

Low quantities of high-quality conversational data Illustration of a woman standing next to a robot, and some example conversation data

Step 3 - RLHF

RLHF = Reinforcement Learning from Human Feedback

Illustration of two women holding question-mark cards, with thought bubbles saying 'Option A' and 'Option B' above their heads

The role of AI in crafting comparison labels

Illustration of a woman giving a thumbs up next to a robot, with three haikus about AI displayed on screens. The woman and robot each have thought bubbles saying 'This one!

The role of AI in crafting comparison labels

Labeling is a human-machine collaboration

Illustration showing a person icon and a robot icon connected by a slider, with a woman and a large robot standing next to them harmoniously

Let's practice!

Intermediate ChatGPT