Date: Wednesday, March 17th, 2021
9:00 am – 10:00 am Pacific Time
12:00 pm – 1:00 pm Eastern Time
Location: Weekly Seminar, Zoom
Title: Closer than they appear: A Bayesian perspective on individual-level heterogeneity in risk assessment
Abstract:
Risk assessment instruments are used across the criminal justice system toestimate the probability of some future behavior given covariates. Theestimated probabilities are then used in making decisions at the individuallevel. In the past, there has been controversy about whether the probabilitiesderived from group-level calculations can meaningfully be applied toindividuals. Using Bayesian hierarchical models applied to a large longitudinaldataset from the court system in the state of Kentucky, we analyze variation inindividual-level probabilities of failing to appear for court and the extent towhich it is captured by covariates. We find that individuals within the samerisk group vary widely in their probability of the outcome. In practice, thismeans that allocating individuals to risk groups based on standard approachesto risk assessment, in large part, results in creating distinctions amongindividuals who are not meaningfully different in terms of their likelihood ofthe outcome. This is because uncertainty about the probability that anyparticular individual will fail to appear is large relative to the differencein average probabilities among any reasonable set of risk groups.
Bio:
James Johndrow is an Assistant Professor in the Statistics department at the University of Pennsylvania. His research interests include high-dimensional statistics, Bayesian computation, algorithmic decision-making, and fairness in ML. Prior to joining Penn, James was a Stein Fellow in the Statistics department at Stanford and a Ph.D. student in Statistical Science at Duke.
Kristian Lum is an Assistant Research Professor in the Department of Computer and Information Science at the University of Pennsylvania. She studies algorithmic (un)fairness, primarily in the use of predictive models in the criminal justice system.