Linear classifiers: prediction equations

Linear Classifiers in Python

Michael (Mike) Gelbart

Instructor, The University of British Columbia

Dot Products

x = np.arange(3)
x

array([0, 1, 2])

y = np.arange(3,6)
y

array([3, 4, 5])

x*y

array([0, 4, 10])

np.sum(x*y)

14

x@y

14
  • x@y is called the dot product of x and y, and is written $x \cdot y$.
Linear Classifiers in Python

Linear classifier prediction

  • $\textrm{raw model output} = \textrm{coefficients} \cdot \textrm{features} + \textrm{intercept}$
  • Linear classifier prediction: compute raw model output, check the sign
    • if positive, predict one class
    • if negative, predict the other class
  • This is the same for logistic regression and linear SVM
    • fit is different but predict is the same
Linear Classifiers in Python

How LogisticRegression makes predictions

$\textrm{raw model output} = \textrm{coefficients} \cdot \textrm{features} + \textrm{intercept}$

lr = LogisticRegression()

lr.fit(X,y)

lr.predict(X)[10]

0

lr.predict(X)[20]

1
Linear Classifiers in Python

How LogisticRegression makes predictions (cont.)

lr.coef_ @ X[10] + lr.intercept_ # raw model output

array([-33.78572166])

lr.coef_ @ X[20] + lr.intercept_ # raw model output

array([ 0.08050621])
Linear Classifiers in Python

The raw model output

Linear Classifiers in Python

The raw model output

Linear Classifiers in Python

The raw model output

Linear Classifiers in Python

Let's practice!

Linear Classifiers in Python

Preparing Video For Download...