Predatory predictions and the ethics of predictive analytics
Appeared In: Journal of the Association for Information Science and Technology
Publication Date: February 16, 2023
In this paper, Martin critically examines ethical issues introduced by predictive analytics. She argues firms can have a market incentive to construct deceptively inflated true-positive outcomes: individuals are over-categorized as requiring a penalizing treatment and the treatment leads to mistakenly thinking this label was correct. Martin shows that differences in power between firms developing and using predictive analytics compared to subjects can lead to firms reaping the benefits of predatory predictions while subjects can bear the brunt of the costs. While profitable, the use of predatory predictions can deceive stakeholders by inflating the measurement of accuracy, diminish the individuality of subjects, and exert arbitrary power. She then argues that firms have a responsibility to distinguish between the treatment effect and predictive power of the predictive analytics program, better internalize the costs of categorizing someone as needing a penalizing treatment, and justify the predictions of subjects and general use of predictive analytics. Subjecting individuals to predatory predictions only for a firms' efficiency and benefit is unethical and an arbitrary exertion of power. Firms developing and deploying a predictive analytics program can benefit from constructing predatory predictions while the cost is borne by the less powerful subjects of the program.
Martin, K. Predatory predictions and the ethics of predictive analytics. Journal of the Association for Information Science and Technology (2023). DOI: 10.1002/asi.24743