| Part of a series on |
| Artificial intelligence |
|---|
_artificial_intelligence_icon.png){kind=icon} |
Algorithmic bias describes systematic and repeatable errors in a computer system that create "unfair" outcomes, such as "privileging" one category over another in ways different from the intended function of the algorithm.
Bias can emerge from many factors, including but not limited to the design of the algorithm or the unintended or unanticipated use or decisions relating to the way data is coded, collected, selected or used to train the algorithm. For example, algorithmic bias has been observed in search engine results and social media platforms. This bias can have impacts ranging from inadvertent privacy violations to reinforcing social biases of race, gender, sexuality, and ethnicity. The study of algorithmic bias is most concerned with algorithms that reflect "systematic and unfair" discrimination.[2] This bias has only recently been addressed in legal frameworks, such as the European Union's General Data Protection Regulation (proposed 2018) and the Artificial Intelligence Act (proposed 2021, approved 2024).
As algorithms expand their ability to organize society, politics, institutions, and behavior, sociologists have become concerned with the ways in which unanticipated output and manipulation of data can impact the physical world. Because algorithms are often considered to be neutral and unbiased, they can inaccurately project greater authority than human expertise (in part due to the psychological phenomenon of automation bias), and in some cases, reliance on algorithms can displace human responsibility for their outcomes. Bias can enter into algorithmic systems as a result of pre-existing cultural, social, or institutional expectations; by how features and labels are chosen; because of technical limitations of their design; or by being used in unanticipated contexts or by audiences who are not considered in the software's initial design.[3]
Algorithmic bias has been cited in cases ranging from election outcomes to the spread of online hate speech. It has also arisen in criminal justice, healthcare, and hiring, compounding existing racial, socioeconomic, and gender biases. The relative inability of facial recognition technology to accurately identify darker-skinned faces has been linked to multiple wrongful arrests of black men, an issue stemming from imbalanced datasets. Problems in understanding, researching, and discovering algorithmic bias persist due to the proprietary nature of algorithms, which are typically treated as trade secrets. Even when full transparency is provided, the complexity of certain algorithms poses a barrier to understanding their functioning. Furthermore, algorithms may change, or respond to input or output in ways that cannot be anticipated or easily reproduced for analysis. In many cases, even within a single website or application, there is no single "algorithm" to examine, but a network of many interrelated programs and data inputs, even between users of the same service.
- ^ Jacobi, Jennifer (September 13, 2001). "Patent #US2001021914". Espacenet. Retrieved July 4, 2018.
- ^ Marabelli, Marco (2024). AI, Ethics, and Discrimination in Business. Palgrave Studies in Equity, Diversity, Inclusion, and Indigenization in Business. Springer. doi:10.1007/978-3-031-53919-0. ISBN 978-3-031-53918-3.
- ^ Suresh, Harini; Guttag, John (November 4, 2021). "A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle". Equity and Access in Algorithms, Mechanisms, and Optimization. EAAMO '21. New York, NY, USA: Association for Computing Machinery. pp. 1–9. doi:10.1145/3465416.3483305. ISBN 978-1-4503-8553-4. S2CID 235436386.