Overfitting e ensembling

Machine Learning per la finanza in Python

Nathan George

Data Science Professor

overfitting

Semplifica il modello

limited net

Opzioni per reti neurali

Opzioni per contrastare l'overfitting:

Riduci il numero di nodi
Usa regularization L1/L2
Dropout
Architettura autoencoder
Early stopping
Aggiungi rumore ai dati
Vincoli max norm
Ensembling

Dropout

dropout

Dropout in keras

from keras.layers import Dense, Dropout

model = Sequential()
model.add(Dense(500,
                input_dim=scaled_train_features.shape[1],
                activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(100, activation='relu'))
model.add(Dense(1, activation='linear'))

Confronto sul test set

Valori R$^2$ su AMD senza dropout:

train: 0,91
test: -0,72

Con dropout:

train: 0,46
test: -0,22

Ensembling

random forest

Implementare l'ensembling

# make predictions from 2 neural net models
test_pred1 = model_1.predict(scaled_test_features)
test_pred2 = model_2.predict(scaled_test_features)

# horizontally stack predictions and take the average across rows
test_preds = np.mean(np.hstack((test_pred1, test_pred2)), axis=1)

Confronto dell'ensemble

Model 1 punteggio R$^2$ sul test set:

-0,179

model 2:

-0,148

ensemble (previsioni medie):

-0,146

Dropout ed ensemble!

Machine Learning per la finanza in Python