Leave-one-out-cross-validation (LOOCV)

Model Validation in Python

Kasey Jones

Data Scientist

LOOCV

LOOCV consists of splitting the data n times. Where n is the number of observations in the data. This takes cross-validation to the extreme. We will have to run n total models.

When to use LOOCV?

Use when:

The amount of training data is limited
You want the absolute best error estimate for new data

Be cautious when:

Computational resources are limited
You have a lot of data
You have a lot of parameters to test

LOOCV Example

n = X.shape[0]
mse = make_scorer(mean_squared_error)
cv_results = cross_val_score(estimator, X, y, scoring=mse, cv=n)

print(cv_results)

[5.45, 10.52, 6.23, 1.98, 11.27, 9.21, 4.65, ... ]

print(cv_results.mean())

6.32

Let's practice

Model Validation in Python