Using loss functions to assess model predictions

Introduction to Deep Learning with PyTorch

Jasmin Ludolf

Senior Data Science Content Developer, DataCamp

Why do we need a loss function?

Tells us how good our model is during training
Takes a model prediction $\hat{y}$ and ground truth $y$
Outputs a float

Loss function diagram

Why do we need a loss function?

Class 0 - mammal, class 1 - bird, class 2 - reptile

Hair	Feathers	Eggs	Milk	Fins	Legs	Tail	Domestic	Catsize	Class
1	0	0	1	0	4	0	0	1	0

Predicted class = 0 -> correct = low loss
Predicted class = 1 -> wrong = high loss
Predicted class = 2 -> wrong = high loss

Our goal is to minimize loss

One-hot encoding concepts

$loss = F(y, \hat{y})$
$y$ is a single integer (class label)
- e.g. $y=0$ when $y$ is a mammal
$\hat{y}$ is a tensor (prediction before softmax)
- If N is the number of classes, e.g. N = 3
- $\hat{y}$ is a tensor with N dimensions,
  - e.g. $\hat{y}$ = [-5.2, 4.6, 0.8]

One-hot encoding concepts

Convert an integer y to a tensor of zeros and ones

One-hot encoding

Transforming labels with one-hot encoding

import torch.nn.functional as F

print(F.one_hot(torch.tensor(0), num_classes = 3))

tensor([1, 0, 0])

print(F.one_hot(torch.tensor(1), num_classes = 3))

tensor([0, 1, 0])

print(F.one_hot(torch.tensor(2), num_classes = 3))

tensor([0, 0, 1])

Cross entropy loss in PyTorch

from torch.nn import CrossEntropyLoss

scores = torch.tensor([-5.2, 4.6, 0.8])
one_hot_target = torch.tensor([1, 0, 0])


criterion = CrossEntropyLoss()

print(criterion(scores.double(), one_hot_target.double()))

tensor(9.8222, dtype=torch.float64)

Bringing it all together

Loss function takes:

scores - model predictions before the final softmax function
one_hot_target - one hot encoded ground truth label

Loss function outputs:

loss - a single float

Loss function diagram with values

Let's practice!

Introduction to Deep Learning with PyTorch