The evolution of generative AI

Generative AI Concepts

Daniel Tedesco

Data Lead, Google

Generative AI burst on the scene in 2023

A chart showing the number of months various software products took to reach 100 million monthly users. ChatGPT is the leader, taking only 2 months.

1 Yahoo Finance
Generative AI Concepts

Key factors driving development

Several factors drive generative AI development:

  • Computing power
  • Dataset availability
  • Competitive interests
  • Model design
Generative AI Concepts

Computational power allowed large models

A chart showing exponential growth of computing power required for AI model training from less than 10^4 floating point operations in 1950 to 10^24 floating point operations in 2022.

 

  • Parallelization and specialized hardware
    • Graphics Processing Units (GPUs)
    • Tensor Processing Units (TPUs)
  • Cloud computing
  • Hardware-software optimization
1 Compute Trends Across Three Eras of Machine Learning, https://arxiv.org/abs/2202.05924
Generative AI Concepts

Models improved with massive datasets

Global Datasphere Growth A chart showing accelerating growth of the global datasphere, from 20 zettabytes in 2015 to 180 zettabytes in 2025.

1 IDC's Global DataSphere, 2021
Generative AI Concepts

Competitive pressures encouraged faster development

Commercial Logos of major technology companies

Political Many different international flags

Generative AI Concepts

GANs unleashed high quality generation

An image of a generator algorithm challenging a discriminator algorithm to classify several confusing pictures of puppies curled up and looking similar to bagels

Generative AI Concepts

Transformers brought context and coherence

'it' refers to 'animal'

The sentence, "The animal didn't cross the street because it was too tired." The words 'it' and 'animal' are highlighted.

'it' refers to 'street'

The sentence, "The animal didn't cross the street because it was too wide." The words 'it' and 'street' are highlighted.

1 https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html
Generative AI Concepts

Transformers brought context and coherence

 

Transformers:

  • Grasp the context of a given text
  • Analyze relationships between words
  • Generate responses that feel natural and informative
Generative AI Concepts

RLHF engaged user feedback

Reinforcement Learning with Human Feedback (RLHF):

  • Reinforcement learning trains models through trial-and-error
  • Human feedback comes from users scoring model responses

A generative AI flow, with an additional loop: response to feedback to model.

Generative AI Concepts

RLHF engaged user feedback

A screenshot of the Midjourney user interface with options for rating an image result.

1 Midjourney
Generative AI Concepts

Let's practice!

Generative AI Concepts

Preparing Video For Download...