Random Forests: Many Classification Trees
Predicting if someone will recidivate is a difficult task. While decision trees are powerful classifying algorithms, they have their advantages and disadvantages. Unlike many machine learning algorithms, decision trees are white-box, meaning the user can see the steps taken in order to classify an individual, and the construction of the tree can be clearly explained; furthermore, they are easy to understand and interpret. Generally, though, decision trees alone are not useful as they are unstable, lack flexibility, and are relatively inaccurate compared to other algorithms.
While the COMPAS algorithm is proprietary, its mechanisms are likely more complex than a standard decision tree. By adopting concepts from decision trees, we can build a more advanced model known as a random forest. Instead of outputting one specific category, a random forest returns a probability of some outcome occurring based on the results of many decision trees. For example, the decision tree for recisivism in the above app will result in either a Yes or No response. Random forest algorithms provide the probability of a person recidivating. When used to predict recidivism, we can use this probabilistic output to create a more equitable model.
A very useful aspect of random forests is that the result is an aggregation of many different, uncorrelated decision trees. Each tree in a random forest is made from bootstrapped data -- a method where a new sample is created by randomly selecting rows from the original sample (with replacement). Another reason random forests produce different trees is because only a subset of variables are considered at each node of a tree’s construction. Since trees are created with bootstrapping and we aggregate the results of many trees to return a probability of an outcome, a random forest is known as a bagging algorithm.
This figure displays how forests reach their predictions
In essence, random forests differ from a single decision tree in three ways:
- Each tree in a random forest is created using a bootstrapped dataset that is the same size as the original.
- Only a subset of variables is considered at each node of a tree.
- Classification of an individual is based on the result of many different, uncorrelated trees.
Note: For more detail about random forests, it is recommended you watch this video here.
After running training or testing data through a model, we can create confusion matrices to examine different accuracy metrics. Based on ProPublica’s report, we can visualize the racial disparities of COMPAS by creating different confusion matrices based on race: