Journey
A Forest of Questions
/journeys/a-forest-of-questions
How machines learn to predict, one yes-or-no question at a time: a single decision tree, then boosting, then random forests, all grown in your browser.
Journey · the map
A Forest of Questions
NNot to scale
Stories2 out · 8 coming
Parts
BY ROADShared start
01The tree that predicts
How can a tree help you guess how many ice creams you'll sell, given the weather?Teaching a tree to stop asking
A decision tree that never stops asking questions ends up memorizing its data. How do we tell it when to stop?Boosting roadBranches off after part 02
03
Many small trees
Small trees, each one fixing the last04
Where the gradient comes from
LightGBM roadBranches off after part 04
05
What makes it “Light”
The fast one, taken apart06
Keeping the model honest
Validation, early stopping, and the leaks that fool both.07
Where trees break
08
Tuning
XGBoost roadBranches off after part 04
05
XGBoost
The same idea, one derivative furtherRandom forests roadBranches off after part 02
03