What is a loss function?

Linear Classifiers in Python

Michael Gelbart

Instructor, The University of British Columbia

Least squares: the squared loss

  • scikit-learn's LinearRegression minimizes a loss: $$\sum_{i=1}^n (\text{true} \; i \text{th target value}-\text{predicted} \; i \text{th target value})^2$$
  • Minimization is with respect to coefficients or parameters of the model.
  • Note that in scikit-learn model.score() isn't necessarily the loss function.
Linear Classifiers in Python

Classification errors: the 0-1 loss

  • Squared loss not appropriate for classification problems (more on this later).
  • A natural loss for classification problem is the number of errors.
  • This is the 0-1 loss: it's 0 for a correct prediction and 1 for an incorrect prediction.
  • But this loss is hard to minimize!
Linear Classifiers in Python

Minimizing a loss

from scipy.optimize import minimize

minimize(np.square, 0).x

array([0.])

minimize(np.square, 2).x

array([-1.88846401e-08])
Linear Classifiers in Python

Let's practice!

Linear Classifiers in Python

Preparing Video For Download...