Veranstaltungen 15. Wissenschaftliche Tagung am 20. und 21. Juni 2024

Datenerhebung, Datenqualität und Datenethik in Zeiten von künstlicher Intelligenz

From Data to Decisions: The Role of Algorithmic Profiling in Shaping Public Policy

Ruben L. Bach

MZES, University of Mannheim; Presenting Author & Christoph Kern, Department of Statistics, LMU


Algorithmic profiling presents an increasingly important avenue for informing high-stakes policy decisions such as the allocation of scarce public resources in a variety of settings. Examples include the allocation of intervention and supervision resources in criminal justice the allocation of in-person investigations in the context of child protection services and the allocation of home inspections to identify and control health hazards. In these scenarios machine learning and statistical models are used to provide an initial risk assessment such that decisions can be made for example regarding the question which cases require special attention and should be prioritized. Likewise prescriptive machine learning is used to identify the optimal treatment for a given case. So far risk assessments and treatment decisions in the government sector are often still based on human experience or on a (small) set of pre-defined rules. The hope is however that statistical and algorithmic profiling will increase both effectiveness and objectivity of the decision-making process. At the same time concerns are raised that statistical profiling may exacerbate existing inequalities and potentially result in unfair and discriminatory decisions.

In this talk we present findings of a series of studies using administrative labor market data to evaluate statistical models for predicting job seekers' risk of becoming long-term unemployed and for predicting the impact of various labor market measures designed to support them in finding a new job. We highlight several key findings from these studies. First we show that different models and modelling choices competitive in predictive performance result in considerable variation in the cases flagged as high risk. Second profiling models can be considerably less accurate for vulnerable social subgroups. That is different classification policies can have very different fairness implications. Third small adjustments to a machine learning based retrospective counterfactual impact evaluation can reduce inequality in outcomes considerably.

We call for rigorous auditing processes before algorithmic profiling models are put to practice and we highlight the need for systematic evaluation and transparency of the full data and analysis pipeline. Algorithmic approaches that promise to take human discretion and biases out of the equation may in fact be far less objective than one might think. Our results can help guide policymakers and inform the public debate on the impacts of algorithmic decision-making.