“…currently quantitative methods are primarily used to justify the status quo. I would argue that they do more harm than good.”
— Narayanan (2022)
Introduction
The use of quantitative methods for assessing discrimination and bias has gained increasing attention in recent years. While some scholars argue that quantitative methods are effective in identifying and addressing discrimination, others, like Narayanan (2022), contend that they serve to justify the status quo and do more harm than good. In this essay, I will examine Narayanan’s position and engage with a number of scholarly sources to discuss the uses, benefits, and downsides of quantitative methods. I will also present my argument and position on Narayanan’s claim.
Narayanan’s Position
Narayanan (2022) argues that quantitative methods for assessing discrimination and bias are primarily used to justify the status quo, and do more harm than good. He posits that statistical models and algorithms are not neutral, but rather reflect the values and assumptions of their creators. As a result, when quantitative methods are used to study social phenomena, they tend to perpetuate existing power structures and social hierarchies. For instance, machine learning algorithms used in hiring or loan approval processes may be biased against certain groups, reflecting the historical discrimination and inequality that exist in society. According to Narayanan, relying on quantitative methods risks reinforcing these existing power structures, rather than challenging them.
The Uses and Benefits of Quantitative Methods
While Narayanan argues that quantitative methods do more harm than good, other scholars have identified various uses and benefits of quantitative methods. One benefit is that statistical models and machine learning algorithms can identify and address disparities in healthcare outcomes. Landon et al. (2003) used statistical methods to show that African American patients are less likely to receive appropriate treatment for cardiovascular disease than white patients. They “conclude that important technical barriers stand in the way of using physician clinical performance assessment for evaluating the competency of individual physicians.” This research has led to efforts to address the underlying causes of these disparities, such as unequal access to healthcare and implicit bias among healthcare providers.
Similarly, Pager and Shepherd (2008) used statistical methods to evaluate the impact of affirmative action policies on employment outcomes. They “provide an overview of major findings from studies of discrimination in each of the four domains [employment, housing, credit markets, and consumer interactions]; and, finally, [they] turn to a discussion of the individual, organizational, and structural mechanisms that may underlie contemporary forms of discrimination.” Their research can help policymakers design more effective policies that promote social justice and reduce discrimination.
Additionally, Caliskan, Bryson, and Narayanan (2017) developed methods for auditing machine learning algorithms to identify sources of bias and discrimination, ensuring that algorithms are fair and unbiased. They showed “that applying machine learning to ordinary human language results in human-like semantic biases,” meaning data could be analyzed for bias. Narayanan was actually one of the authors in this paper, which shows how this might have influenced his opinions.
Lastly, Crooks and Heppenstall (2012) also suggest that quantitative methods can be used to analyze and understand complex urban systems. They argue that these methods can contribute to the development of more inclusive and sustainable cities, stating that it “is powerful paradigm that holds great promise for facilitating greater understanding of geographical systems.” They suggest that quantitative methods can be combined with qualitative methods to provide a more comprehensive understanding of urban phenomena.
More Support
Diving deeper into their study (Pager and Shepherd (2008)) on discrimination in hiring, Pager and Shepherd found how affirmative action policies resulted in significant improvements in the employment outcomes of African American and Latino candidates. The policies helped to reduce discrimination in hiring and promote social justice. Similarly, Smith and Hasan (2020) argues that quantitative methods can be used to promote fairness in algorithmic decision-making. In their research, it had far-reaching concequences in psychiatric patients whose background could be well understood by their model. By defining fairness mathematically and incorporating it into algorithms, we can reduce bias and promote social justice.
In their notable investigation, Dressel and Farid (2018) scrutinized the accuracy of COMPAS, a widely utilized commercial algorithm employed in the realm of criminal justice decision-making. The study brought to light a troubling revelation: the algorithm exhibited bias against African American defendants, as they were disproportionately categorized as high-risk compared to their white counterparts. This finding not only raises significant concerns about the fairness and equity of the criminal justice system but also underscores the urgent necessity for enhanced transparency and accountability in algorithmic decision-making.
The ramifications of biased algorithms in criminal justice decision-making are profound. The overrepresentation of African American defendants classified as high-risk could potentially lead to harsher sentences, increased pretrial detention, and a perpetuation of existing racial disparities within the system. Such biased outcomes deepen social inequalities and undermine public trust in the fairness of the justice system.
To address these challenges, it is imperative to establish greater transparency in algorithmic decision-making processes. This entails a comprehensive understanding of how algorithms are developed, the data they are trained on, and the potential biases embedded within them. The lack of transparency surrounding proprietary algorithms, like COMPAS, has raised concerns about the potential opacity and inscrutability of these systems.
Accountability is equally crucial. There should be mechanisms in place to assess the performance and impact of algorithms in real-world contexts. This includes conducting regular audits, evaluating the accuracy and fairness of algorithmic outcomes, and addressing any identified biases promptly. Establishing clear guidelines and standards for the use of algorithms in the criminal justice system can help mitigate the potential harm caused by biased decision-making.
Lamont and Molnár (2002) suggest that quantitative methods can reinforce existing power structures and ideologies. They argue that quantitative methods tend to prioritize “objective” and “scientific” knowledge, which is often based on dominant cultural assumptions and perspectives. This approach can lead to the exclusion of diverse perspectives and the marginalization of underrepresented groups.
Finally, Blodgett et al. (2020) argue that quantitative methods can be used to address historical inequalities and promote social justice. They argue that by collecting and analyzing data on the experiences of marginalized groups, we can identify and address systemic inequalities. This approach can help to promote social justice and reduce discrimination.
Argument and Position
In conclusion, while Narayanan (2022) argues that quantitative methods do more harm than good, the scholarly sources reviewed in this essay demonstrate that quantitative methods can be effective in identifying and addressing discrimination. While there are instances where quantitative methods may serve to justify the status quo and perpetuate existing power structures, they can also be used to promote social justice and reduce discrimination. The key, like for many things in life, is to take into account all factors. This is easier said than done.
Conclusion
Narayanan’s argument that quantitative methods are primarily used to justify the status quo and do more harm than good is a valid concern. However, it does not mean that quantitative methods should not be used in the analysis of discrimination and bias. Rather, it highlights the need for a critical examination of the way these methods are used and the context in which they are applied.
Quantitative methods are a valuable tool in uncovering patterns of discrimination and bias that may be hidden in complex data. As a result, they can be used to hold institutions and individuals accountable for their actions. However, the analysis of discrimination and bias should not be reduced to a simple numbers game, where the goal is to achieve statistical parity. It is important to recognize that discrimination and bias are systemic issues that require a more comprehensive approach.
Therefore, it is essential to consider the limitations of quantitative methods and to use them in conjunction with qualitative methods and other forms of analysis to gain a more holistic understanding of the problem. In doing so, we can ensure that we are not only identifying patterns of discrimination and bias but also developing effective strategies to address them.
In conclusion, Narayanan’s assertion that quantitative methods can do more harm than good is an important reminder that we need to be mindful of the ways in which we use these methods. While quantitative methods can be a valuable tool in uncovering patterns of discrimination and bias, they are not a panacea. To fully understand and address discrimination and bias, we need to take a more comprehensive approach that includes both quantitative and qualitative methods, as well as a critical examination of the social and political contexts in which discrimination and bias occur.