The strength of "weak" models

Ensemble Methods in Python

Román de las Heras

Data Scientist, Appodeal

"Weak" model

Voting and Averaging:

  • Small number of estimators
  • Fine-tuned estimators
  • Individually trained

New concept: "weak" estimator

Ensemble Methods in Python

weak vs strong.jpeg

Ensemble Methods in Python

Properties of "weak" models

Weak estimator

  • Performance better than random guessing
  • Light model
  • Low training and evaluation time

Example: Decision Tree

decision_tree_weak.bmp

Ensemble Methods in Python

Examples of "weak" models

Some "weak" models:

  • Decision tree: small depth

  • Logistic Regression

  • Linear Regression

  • Other restricted models

Sample code:

model = DecisionTreeClassifier(
    max_depth=3
)

model = LogisticRegression(
    max_iter=50, C=100.0
)

model = LinearRegression()
Ensemble Methods in Python

Let's practice!

Ensemble Methods in Python

Preparing Video For Download...