Loss functions

Introduction to TensorFlow in Python

Isaiah Hull

Visiting Associate Professor of Finance, BI Norwegian Business School

Introduction to loss functions

Fundamental tensorflow operation
- Used to train a model
- Measure of model fit
Higher value -> worse fit
- Minimize the loss function

Common loss functions in TensorFlow

TensorFlow has operations for common loss functions
- Mean squared error (MSE)
- Mean absolute error (MAE)
- Huber error
Loss functions are accessible from tf.keras.losses()
- tf.keras.losses.mse()
- tf.keras.losses.mae()
- tf.keras.losses.Huber()

Why do we care about loss functions?

This image shows the mse, mae, and huber loss functions plotted over the -2 to 2 interval.

MSE
- Strongly penalizes outliers
- High (gradient) sensitivity near minimum
MAE
- Scales linearly with size of error
- Low sensitivity near minimum
Huber
- Similar to MSE near minimum
- Similar to MAE away from minimum

Defining a loss function

# Import TensorFlow under standard alias
import tensorflow as tf

# Compute the MSE loss
loss = tf.keras.losses.mse(targets, predictions)

Defining a loss function

# Define a linear regression model
def linear_regression(intercept, slope = slope, features = features):
    return intercept + features*slope

# Define a loss function to compute the MSE
def loss_function(intercept, slope, targets = targets, features = features):
    # Compute the predictions for a linear model
    predictions = linear_regression(intercept, slope)

    # Return the loss
    return tf.keras.losses.mse(targets, predictions)

Defining the loss function

# Compute the loss for test data inputs
loss_function(intercept, slope, test_targets, test_features)

10.77

# Compute the loss for default data inputs
loss_function(intercept, slope)

5.43

Let's practice!

Introduction to TensorFlow in Python