Governing algorithmic decisions: The role of decision importance and governance on perceived legitimacy of algorithmic decisions
Appeared In: Big Data & Society
Publication Date: June 28, 2022
The algorithmic accountability literature to date has primarily focused on procedural tools to govern automated decision-making systems. That prescriptive literature elides a fundamentally empirical question: Whether and under what circumstances, if any, is the use of algorithmic systems to make public policy decisions perceived as legitimate? The present study begins to answer this question.
Using factorial vignette survey methodology, Martin and Waldman explore the relative importance of the type of decision, the procedural governance, the input data used, and outcome errors on perceptions of the legitimacy of algorithmic public policy decisions as compared to similar human decisions. Among other findings, they find that the type of decision—low importance versus high importance—impacts the perceived legitimacy of automated decisions. The authors find that human governance of algorithmic systems (aka human-in-the-loop) increases perceptions of the legitimacy of algorithmic decision-making systems, even when those decisions are likely to result in significant errors. Notably, they also find the penalty to perceived legitimacy is greater when human decision-makers make mistakes than when algorithmic systems make the same errors. The positive impact on perceived legitimacy from governance—such as human-in-the-loop—is greatest for highly pivotal decisions such as parole, policing, and healthcare. After discussing the study’s limitations, Martin and Waldman outline avenues for future research.
Martin, K., Waldman, A. Governing algorithmic decisions: The role of decision importance and governance on perceived legitimacy of algorithmic decisions. Big Data & Society (2022). DOI: 10.1177/20539517221100449