The evolution of generative AI

Concetti di IA generativa

Daniel Tedesco

Data Lead, Google

Generative AI burst on the scene in 2023

A chart showing the number of months various software products took to reach 100 million monthly users. ChatGPT is the leader, taking only 2 months.

1 Yahoo Finance
Concetti di IA generativa

Key factors driving development

Several factors drive generative AI development:

  • Computing power
  • Dataset availability
  • Competitive interests
  • Model design
Concetti di IA generativa

Computational power allowed large models

A chart showing exponential growth of computing power required for AI model training from less than 10^4 floating point operations in 1950 to 10^24 floating point operations in 2022.

 

  • Parallelization and specialized hardware
    • Graphics Processing Units (GPUs)
    • Tensor Processing Units (TPUs)
  • Cloud computing
  • Hardware-software optimization
1 Compute Trends Across Three Eras of Machine Learning, https://arxiv.org/abs/2202.05924
Concetti di IA generativa

Models improved with massive datasets

Global Datasphere Growth A chart showing accelerating growth of the global datasphere, from 20 zettabytes in 2015 to 180 zettabytes in 2025.

1 IDC's Global DataSphere, 2021
Concetti di IA generativa

Competitive pressures encouraged faster development

Commercial Logos of major technology companies

Political Many different international flags

Concetti di IA generativa

GANs unleashed high quality generation

An image of a generator algorithm challenging a discriminator algorithm to classify several confusing pictures of puppies curled up and looking similar to bagels

Concetti di IA generativa

Transformers brought context and coherence

'it' refers to 'animal'

The sentence, "The animal didn't cross the street because it was too tired." The words 'it' and 'animal' are highlighted.

'it' refers to 'street'

The sentence, "The animal didn't cross the street because it was too wide." The words 'it' and 'street' are highlighted.

1 https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html
Concetti di IA generativa

Transformers brought context and coherence

 

Transformers:

  • Grasp the context of a given text
  • Analyze relationships between words
  • Generate responses that feel natural and informative
Concetti di IA generativa

RLHF engaged user feedback

Reinforcement Learning with Human Feedback (RLHF):

  • Reinforcement learning trains models through trial-and-error
  • Human feedback comes from users scoring model responses

A generative AI flow, with an additional loop: response to feedback to model.

Concetti di IA generativa

RLHF engaged user feedback

A screenshot of the Midjourney user interface with options for rating an image result.

1 Midjourney
Concetti di IA generativa

Let's practice!

Concetti di IA generativa

Preparing Video For Download...