Random Forest

Uses many decision trees to make better predictions. Each tree looks at different random parts of the data and their results are combined by voting for classification or averaging for regression.

• Using random data and features for each tree avoid over-fitting and makes the overall prediction more accurate.

• Handles missing data

• Shows feature importance
• Works well with big and complex data
• Doesn’t require normalisation or standardisation on dataset

Cons

• It can be computationally expensive
• Harder to interpret compared to other models

Individual Decision Tree

Each tree uses a mathematical metric called Gini Impurity to decide the best split

G = 1 - sum(pi)^2

pi → probability of a file being in class i

Picks the split that minimises the gini score

Tunable Parameters

• Number of Trees

As you add more trees, the variance of the model decreases, but increases computational time

• Maximum Depth

Low depth cannot answer complex patterns, very deep trees may memorise the training data.

• Minimum Samples split

This sets the requirement for a node to split into branches. If we set this to 10, a node must have atleast 10 data points inside it before it is allowed to split further.

Higher values prevent the tree from creating branches based on tiny, insignificant groups of data. So can avoid “weird tweets”

• Minimum samples leaf

Defines the minimum number of data points that must exist in a Leaf. It ensures that every prediction the forest makes is based on average of at least a few samples

Parameter	To Reduce Overfitting...	To Reduce Underfitting...
Number of Trees	Increase it (more stability)	Increase it
Max Depth	Decrease it (shorter trees)	Increase it
Min Samples Split	Increase it	Decrease it
Min Samples Leaf	Increase it	Decrease it