Explainability in linear models

Explainable AI in Python

Fouad Trad

Machine Learning Engineer

Linear models

Linear regression

Predicts continuous values

Image showing a line that fits multiple points, where the x-axis represents a test score and y-axis represents the chance of admission.

Linear models

Linear regression

Predicts continuous values

Image showing a line that fits multiple points, where the x-axis represents a test score and y-axis represents the chance of admission.

Logistic regression

Used for binary classification

Image showing a line that separates multiple data points that represent transactions based on the transaction amount and the time it was executed.

Why are linear models explainable?

Learn linear combination of input features
$c_0 + c_1 \times \text{feature}_1 + c_2 \times \text{feature}_2 + \ldots + c_n \times \text{feature}_n$

Image showing the same linear and logistic regression graphs as before, but with the equation that each learns: linear regression learns an equation of the form Mobile price=?_0+?_1?Storage, and logistic regression learns an equation of the form ?_0+?_1?Time of day+?_2?Transaction amount=0

Coefficients

Tell us importance of each feature
- Higher absolute value → higher importance
- Lower absolute value → lower importance
To compare coefficients → absolute values
Note: Normalize feature scales before computing coefficients

Image showing the equation Mobile price=?_0+5?Storage+2?Number of lenses, highlighting the coefficients

Coefficients

Tell us importance of each feature
- Higher absolute value → higher importance
- Lower absolute value → lower importance
To compare coefficients → absolute values
Note: Normalize feature scales before computing coefficients

Image showing the equation Mobile price=?_0+?_1?Storage+c_2?Number of lenses, highlighting that Storage sizes are typically in ranges of hundreds to thousands Gigabytes, and number of lenses is usually less than 4.

Admissions

GRE Score	TOEFL Score	University Rating	SOP	LOR	CGPA	Chance of Admit	Accept
337	118	4	4.5	4.5	9.65	0.92	1
324	107	4	4	4.5	8.87	0.76	1
316	104	3	3	3.5	8	0.72	1
322	110	3	3.5	2.5	8.67	0.8	1
314	103	2	2	3	8.21	0.45	0

X_train = data.drop(['Chance of Admit', 'Accept'], axis=1)

y_reg = data['Chance of Admit']

y_cls = data['Accept']

Model training

from sklearn.preprocessing import MinMaxScaler
from sklearn.linear_model import LinearRegression, LogisticRegression


scaler = MinMaxScaler()
X_train_scaled = scaler.fit_transform(X_train)


lin_reg = LinearRegression()
lin_reg.fit(X_train_scaled, y_reg)


log_reg = LogisticRegression()
log_reg.fit(X_train_scaled, y_cls)

Computing coefficients

Linear regression

print(lin_reg.coef_)

[0.03052087 0.01665433 0.00668971 
 0.00326926 0.01724815 0.0661691 ]

Logistic regression

print(log_reg.coef_)

[[1.28985577  0.49441086  0.47593379 
  0.05434322  0.41800927  1.31980189]]

Visualizing coefficients

Linear regression

import matplotlib.pyplot as plt
plt.bar(X_train.columns, lin_reg.coef_)

A bar plot of the importances showing that CGPA and GRE score are the most important features.

Logistic regression

import matplotlib.pyplot as plt
plt.bar(X_train.columns, log_reg.coef_[0])

A bar plot of the importances showing that CGPA and GRE score are the most important features.

Let's practice!

Explainable AI in Python