Target-Encoding
Eine Kaggle-Competition in Python gewinnen
Yauhen Babakhin
Kaggle Grandmaster
Kategoriale Features mit hoher Kardinalität
- Label-Encoder gibt jeder Kategorie eine eindeutige Zahl
- One-Hot-Encoder erstellt für jeden Kategorienwert ein neues Feature
- Target-Encoding hilft hier!
Mean-Target-Encoding
| Train-ID | Kategorial | Target |
|---|---|---|
| 1 | A | 1 |
| 2 | B | 0 |
| 3 | B | 0 |
| 4 | A | 1 |
| 5 | B | 0 |
| 6 | A | 0 |
| 7 | B | 1 |
| Test-ID | Kategorial | Target |
|---|---|---|
| 10 | A | ? |
| 11 | A | ? |
| 12 | B | ? |
| 13 | A | ? |
Mean-Target-Encoding
- Mittelwert im Train berechnen, auf Test anwenden
- Train in K Folds splitten. Mittelwert auf (K-1) Folds berechnen, auf den K-ten anwenden
- Mean-Target-Feature dem Modell hinzufügen
Mittelwert im Train berechnen
| Train-ID | Kategorial | Target |
|---|---|---|
| 1 | A | 1 |
| 2 | B | 0 |
| 3 | B | 0 |
| 4 | A | 1 |
| 5 | B | 0 |
| 6 | A | 0 |
| 7 | B | 1 |
Mittelwert im Train berechnen
| Train-ID | Kategorial | Target |
|---|---|---|
| 1 | A | 1 |
| 2 | B | 0 |
| 3 | B | 0 |
| 4 | A | 1 |
| 5 | B | 0 |
| 6 | A | 0 |
| 7 | B | 1 |
Mittelwert im Train berechnen
| Train-ID | Kategorial | Target |
|---|---|---|
| 1 | A | 1 |
| 2 | B | 0 |
| 3 | B | 0 |
| 4 | A | 1 |
| 5 | B | 0 |
| 6 | A | 0 |
| 7 | B | 1 |
Test-Encoding
| Test-ID | Kategorial | Target | Mean-Encoded |
|---|---|---|---|
| 10 | A | ? | 0,66 |
| 11 | A | ? | 0,66 |
| 12 | B | ? | 0,25 |
| 13 | A | ? | 0,66 |
Train-Encoding mit Out-of-Fold
| Train-ID | Kategorial | Target | Fold |
|---|---|---|---|
| 1 | A | 1 | 1 |
| 2 | B | 0 | 1 |
| 3 | B | 0 | 1 |
| 4 | A | 1 | 1 |
| 5 | B | 0 | 2 |
| 6 | A | 0 | 2 |
| 7 | B | 1 | 2 |
Train-Encoding mit Out-of-Fold
| Train-ID | Kategorial | Target | Fold | Mean-Encoded |
|---|---|---|---|---|
| 1 | A | 1 | 1 | |
| 2 | B | 0 | 1 | |
| 3 | B | 0 | 1 | |
| 4 | A | 1 | 1 | |
| 5 | B | 0 | 2 | |
| 6 | A | 0 | 2 | |
| 7 | B | 1 | 2 |
Train-Encoding mit Out-of-Fold
| Train-ID | Kategorial | Target | Fold | Mean-Encoded |
|---|---|---|---|---|
| 1 | A | 1 | 1 | 0 |
| 2 | B | 0 | 1 | 0,5 |
| 3 | B | 0 | 1 | 0,5 |
| 4 | A | 1 | 1 | 0 |
| 5 | B | 0 | 2 | |
| 6 | A | 0 | 2 | |
| 7 | B | 1 | 2 |
Train-Encoding mit Out-of-Fold
| Train-ID | Kategorial | Target | Fold | Mean-Encoded |
|---|---|---|---|---|
| 1 | A | 1 | 1 | 0 |
| 2 | B | 0 | 1 | 0,5 |
| 3 | B | 0 | 1 | 0,5 |
| 4 | A | 1 | 1 | 0 |
| 5 | B | 0 | 2 | |
| 6 | A | 0 | 2 | |
| 7 | B | 1 | 2 |
Train-Encoding mit Out-of-Fold
| Train-ID | Kategorial | Target | Fold | Mean-Encoded |
|---|---|---|---|---|
| 1 | A | 1 | 1 | 0 |
| 2 | B | 0 | 1 | 0,5 |
| 3 | B | 0 | 1 | 0,5 |
| 4 | A | 1 | 1 | 0 |
| 5 | B | 0 | 2 | 0 |
| 6 | A | 0 | 2 | 1 |
| 7 | B | 1 | 2 | 0 |
Praxisleitfäden
Praxisleitfäden
Smoothing
$$mean\_enc_i = \frac{target\_sum_i}{n_i}$$
$$smoothed\_mean\_enc_i = \frac{target\_sum_i + \alpha*global\_mean}{n_i + \alpha}$$
$$\alpha \in [5; 10]$$
Praxisleitfäden
Smoothing
$$mean\_enc_i = \frac{target\_sum_i}{n_i}$$
$$smoothed\_mean\_enc_i = \frac{target\_sum_i + \alpha*global\_mean}{n_i + \alpha}$$
$$\alpha \in [5; 10]$$
Neue Kategorien
- Neue Kategorien im Test mit dem global_mean füllen
Praxisleitfäden
| Train-ID | Kategorial | Target |
|---|---|---|
| 1 | A | 1 |
| 2 | B | 0 |
| 3 | B | 0 |
| 4 | A | 0 |
| 5 | B | 1 |
| Test-ID | Kategorial | Target | Mean-Encoded |
|---|---|---|---|
| 10 | A | ? | 0,43 |
| 11 | B | ? | 0,38 |
| 12 | C | ? | 0,40 |
Lass uns üben!
Eine Kaggle-Competition in Python gewinnen
