ProPublica’s investigation into COMPAS, the Correctional Offender Management Profiling for Alternative Sanctions algorithm, revealed a system fraught with racial bias and technical limitations that undermine its role in the criminal justice system. The investigation, published in 2016, demonstrated that the risk assessment tool, widely adopted by courts across the United States, was no more accurate than a simple checklist in predicting violent crime, while falsely flagging black defendants as high risk at twice the rate of white defendants.
How COMPAS Risk Assessment Works
The COMPAS algorithm, developed by the Northpointe company (now Equivant), analyzes static data points from a defendant’s criminal history, age, and demographics to generate a score intended to gauge the likelihood of future criminality. This score, presented as a low, medium, or high risk, often influences judicial decisions regarding bail, sentencing, and probation. ProPublica’s analysis questioned the validity of these classifications, arguing that the proprietary formula lacks transparency and scientific rigor, making it difficult to verify its accuracy or fairness.
Key Findings from the ProPublica Investigation
Journalists Julia Angwin, Jeff Larson, and others meticulously compared COMPAS predictions with actual outcomes for over 10,000 individuals arrested in one Florida county. Their statistical analysis showed that the algorithm’s error rates were significant and unevenly distributed. While it correctly identified low-risk individuals, it systematically overestimated the risk posed by black defendants, potentially perpetuating existing disparities in incarceration rates.
Accuracy and Racial Disparities
The core criticism leveled by ProPublica was that COMPAS failed the basic standard of predictive parity across racial groups. The tool was found to be only slightly more accurate than a coin toss for violent offenses. The false positive rate for black defendants was substantially higher, meaning thousands of individuals deemed high risk posed no more threat than those classified as low risk, yet faced harsher treatment within the system. This discrepancy raises profound ethical questions about the deployment of such technology in a justice system already struggling with bias.
Legal and Ethical Repercussions
The publication of the investigation triggered widespread debate and legal challenges. Defense attorneys argued that the secretive nature of the algorithm violated due process rights, as defendants could not effectively challenge a tool they could not see or understand. Several courts began to scrutinize the use of COMPAS, with some ruling that the proprietary nature of the software was incompatible with the Sixth Amendment right to confront evidence. The findings prompted calls for greater transparency and regulation of algorithmic decision-making in the judiciary.
Industry Response and System Evolution
Equivant, the company behind COMPAS, defended its methodology, stating that the tool was not designed to predict violence specifically and that its validity had been established in peer-reviewed research. They emphasized that the scores were intended to assist, not replace, judicial judgment. Despite the controversy, the use of algorithmic risk assessments has continued to grow, leading to the development of competing tools and updated versions of COMPAS that claim to address some of the identified biases, although independent verification remains a challenge.
The Lasting Impact on Criminal Justice Reform
The ProPublica COMPAS investigation remains a landmark case in data journalism, fundamentally altering the conversation around technology and justice. It forced policymakers, legal scholars, and the public to confront the limitations of algorithmic objectivity. The work highlighted the urgent need for standardized auditing processes, accessible explanations of algorithmic logic, and robust legal frameworks to ensure that these tools reduce, rather than reinforce, systemic inequality within the criminal justice system.
