Forecasting the Supreme Court: A Comparative Analysis of Machine Learning Algorithms on Petitioner vs. Appellee Outcomes
Loading...
Links to Files
Permanent Link
Author/Creator
Author/Creator ORCID
Date
2024-04-25
Department
Hood College Department of Computer Science and Information Technology
Program
Hood College Departmental Honors
Citation of Original Publication
Rights
This thesis is the intellectual property of Benjamin Chase Davids and has been submitted to the Department of Computer Science and Information Technology and the Honors Department at Hood College in partial fulfillment of the requirements for the computer science and departmental honors program. This thesis is available for unrestricted access and dissemination under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) 4.0 International License (https://creativecommons.org/). You are free to share and adapt the material, but you must give appropriate credit, not use it for commercial purposes, and share any derivative works under the same license.
Attribution-NonCommercial-ShareAlike 3.0 United States
Attribution-NonCommercial-ShareAlike 3.0 United States
Abstract
Since its inception, Supreme Court decisions have impacted American laws and life. The ability to predict the high court’s decisions, known as quantitative legal prediction, would be of interest to those in the legal profession and the general public. While much research has been conducted on quantitative legal prediction, for various foreign high courts, the few experiments that have specifically addressed United States Supreme Court cases are now outdated, have been prone to overfitting, or were based on limited datasets. Our work and experimentation attempt to predict case outcomes while addressing the shortcomings of past research. In this work, we deployed several machine learning algorithms to predict whether the petitioner or appellee will win a Supreme Court case and compared the algorithms based on their prediction accuracy. Finally, we embarked on identifying which case features have the greatest predictive impact on the winner of a case. Using four machine learning algorithms (Random Forest, XGBoost, LightGBM, and Multilayer Perceptron) we trained, evaluated, and tested the predictive accuracy on the Washington University School of Law dataset of over 8,000 Supreme Court cases that were litigated between 1946 to 2016. Success was measured via a model’s accuracy, AUROC, and the associated weighted F1 score. Three of the four algorithms achieved accuracy, AUROC, and weighted F1 score in the mid-0.70s with LightGBM being the most accurate. The three case features that most influence LightGBM’s performance are the reason the Supreme Court granted a petition for certiorari, the category of the appellee, and the category of the petitioner. High performing algorithms and models such as the ones we have deployed could provide some predictive insight to individuals, lawyers, and policymakers that may be affected by Supreme Court decisions. Future research directions may include training the algorithms using semantically meaningful textual data or additional case variables.