Evaluating and mitigating social bias

Generative AI Concepts

Daniel Tedesco

Data Lead, Google

What do we mean by social bias?

$$ $$

Systematic unfairness in generative AI

  • Serious societal consequences
  • Fairness can be subjective
  • Focus on broadly shared values

$$ $$

A job interview.

Generative AI Concepts

Where bias appears

  • Training data
  • The model itself
  • How the model is used

Unbalanced weighing scales

Generative AI Concepts

Bias in data

Skewed or unrepresentative information in the training dataset

Two training funnels. One shows purple squares going into training and only purple squares being generated. The other shows a variety of colored shapes going into training and a variety of shapes being generated

Generative AI Concepts

Bias in models

Pursuing goals that result in biased outcomes

Photorealistic robot politician giving a speech at the UN

Generative AI Concepts

Bias in use

Applying AI in wrong or malicious ways

Laptop showing a video with a giant "FAKE" label on it.

Generative AI Concepts

Identifying bias in data and models

  • Representation analysis compares how the model refers to different groups
  • Fairness metrics evaluate models for equal treatment, opportunity, and accuracy across groups
  • Human audits ask real people to review a model's outputs to identify bias
Generative AI Concepts

Mitigating bias in data and models

 

  • Diversify data collection
  • Adjust model to prioritize different data
  • Adversarial training
  • Continuous improvement

A collection of shapes of different colors and a representative sample of them selected

Generative AI Concepts

Let's practice!

Generative AI Concepts

Preparing Video For Download...