Random Forest - Hyperparameters And Tuning

Hyperparameters

Parameter	Description	Typical Values	Effect
n_estimators	Number of trees	100-500	More trees → better performance, slower
max_depth	Maximum tree depth	None or 10-30	Lower → less overfitting
min_samples_split	Min samples to split	2-10	Higher → simpler trees
min_samples_leaf	Min samples per leaf	1-5	Higher → smoother boundaries
max_features	Features per split	’sqrt’, ‘log2’, or number	Lower → more diversity
bootstrap	Use bootstrap samples	True	False uses all data
oob_score	Use OOB for validation	False	True calculates OOB error

Tuning Strategy

1. Start with defaults:

   • n_estimators=100
   • max_features=‘sqrt’ (classification) or ‘auto’ (regression)

2. Increase n_estimators:

   • More trees almost always helps
   • Stop when performance plateaus

3. Tune max_features:

   • Lower values increase diversity
   • Try sqrt(n), log2(n), n/3

4. Adjust tree complexity:

   • max_depth, min_samples_split, min_samples_leaf
   • Use to prevent overfitting if needed