Comparing Grid and Random Search
Hyperparameter Tuning in Python
Alex Scriven
Data Scientist
What's the same?
Similarities between Random and Grid Search?
- Both are automated ways of tuning different hyperparameters
- For both you set the grid to sample from (which hyperparameters and values for each)
Remember to think carefully about your grid!
- For both you set a cross-validation scheme and scoring function
What's different?
Grid Search:
- Exhaustively tries all combinations within the sample space
- No Sampling methodology
- More computationally expensive
- Guaranteed to find the best score in the sample space
Random Search:
- Randomly selects a subset of combinations within the sample space (that you must specify)
- Can select a sampling methodology (other than uniform which is default)
- Less computationally expensive
- Not guaranteed to find the best score in the sample space (but likely to find a good one faster)
Which should I use?
So which one should I use? What are my considerations?
- How much data do you have?
- How many hyperparameters and values do you want to tune?
- How much resources do you have? (Time, computing power)
- More data means random search may be better option.
- More of these means random search may be a better option.
- Less resources means random search may be a better option.
Let's practice!
Hyperparameter Tuning in Python
