The Potential for Bias in Risk-Assessment Tools: A Conversation

In this New Thinking podcast, Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson discuss some of the criticisms that have been leveled against risk assessment tools. Those criticisms include placing too much emphasis on geography and criminal history, which can distort the actual risk for clients from neighborhoods that experience an above-average presence of policing and social services. “Geography is often a proxy for race,” Miller says. Miller and Crosby-Robinson spoke with the Center for Court Innovation’s Director of Communications Robert V. Wolf after they participated in a panel on the “The Risk-Needs-Responsivity Framework”  at Justice Innovation in Times of Change, a regional summit on Sept. 30, 2016 in North Haven, Conn. Reuben J. Miller, assistant professor of social work at the University of Michigan, and his research collaborator Hazelette Crosby-Robinson participate in a panel at “Justice Innovation in Times of Change,” a regional summit. WOLF: Hi, I’m Rob Wolf, Director of Communications at the Center for Court Innovation and today with me at the Justice Innovation in Times of Change Conference here at the Quinnipiac School of Law in North Haven, Connecticut are two of the panelists who participated in a discussion about risk needs assessment tools. They are Professor Reuben Miller, who is an assistant professor of social work at the School of Social Work at the University of Michigan and his research assistant at the School of Social Work, Hazelette Crosby-Robinson.  Thank you so much for taking the time after your panel to sit down and talk with me. MILLER: Thank you for having us. WOLF: So, I wanted to just start off talking about the risk assessment tools and some of the criticisms that have been leveled against them because, as we heard on the panel from Sarah Fritsche, a colleague of mine at the Center for Court Innovation, their use has exploded and they’ve been embraced as a decision-making tool in the criminal justice setting. MILLER: Sure. WOLF: But you raised some potential concerns about them and some of their limitations and I wondered if you could share what some of those limitations are as you see them. MILLER: Sure, I’m happy to. So, Hazelette is my research associate and collaborator. She’s super modest. So, I’d like to first preface this by saying, some scholars have suggested that we’ve really entered an actuarial age. So it’s not just risk assessment in criminal justice, but a whole cost benefits calculus, a whole risk calculus that’s based on actuarial models that try to predict future harm. So they try to predict, much like an insurance company would try to predict the future risk of a car accident. In a criminal justice setting, these risk needs assessments are trying to, one, gauge the needs of incarcerated individuals or people who have been convicted of a crime to try to figure out where they could shore up deficits in their skill sets or in their general stability. So for example, they might examine things like housing stability, or whether or not one was employed, or what kinds of service needs they may have. So for example, if one has a history of substance use and abuse, that would indicate that they need treatment or some sort of intervention based around these things. And at the same time, they’re trying to gauge the risk of re-offense, so the risk that they will commit a crime. So there are a number of criticisms. The literature that engages this is fairly long. I tend to think about some of the movers and shakers in this field, Kelly Hannah-Moffat, Bernard Harcourt, Sonia Star, Faye Taxman. Faye Taxman’s work is actually helping us to think about important ways that we can implement risk assessment that reduce some of the biases that are sort of baked into it, but just to talk about some of the critiques that have come from this literature and of course my own, on the one hand there are static factors like where one lives, so geography, their prior criminal history. These are things that they can’t avoid. And the privileging of recidivism as an indicator of success. These are all problematic for the following reasons. So geography is often a proxy for race. We know that we live in a country that has a pattern of residential racial segregation. And we know that policing and criminal justice resources of all kinds are overwhelmingly distributed in areas where poor people of color tend to live. The problem is, people are now being arrested from, returned to, and even given programs designed to rehabilitate them all within low income communities. Very bounded geographic districts. And so what you get is, you get the overwhelming concentration of criminal justice resources, and you get a signaling of what that all means. So if the substance abuse treatment house is located in a neighborhood, then that tells me that there’s substance abusers there. Right? And so that signals narcotics forces to the community. It says something about the community. Halfway houses are also overwhelmingly there. And so one must think about what the concentration of these things do. So now okay, as it relates to risk. Being in a neighborhood like this triggers a higher risk score. It is indeed one of the measures of risk, and so in that way it’s a proxy for race. Sorry, I know I’m talking quite a bit, but – WOLF: No, and just to kind of summarize though, or to recap what you’ve said so far, the way risk assessment tools work, they place a high value on the location someone’s from. They place a high value on their history with arrest. MILLER: That’s absolutely right. WOLF: And so, if there’s a preponderance of enforcement there, some people are more likely to have an arrest record or – MILLER: The study from Stop and Frisk made this abundantly clear. That even when people aren’t doing anything wrong they’re being overwhelmingly stopped if they’re black or Latino. And so we know that criminal justice contact increases the likelihood that one will be arrested. And so anyway, this is a big problem of using prior arrest records for example and even prior conviction records, so now you’ve got a bunch of arrests. By the time you get to the prosecuting attorney, they’re going to say, Look, you’ve been arrested 14 times. “Well, I’ve been arrested 14 times but never charged.” No, but you have a history of arrest, and so I’m going to now charge you because I see a pattern. This is how statistical discrimination might work, or does in fact work in practice. So now the prosecuting attorney sees a pattern. Sends it before the judge, who looks at this pattern and interprets it to make a decision about the length of the sentence when the conviction is read, as is a jury if it ever goes to trial. 97% of cases never go to trial, but when it goes to trial, juries are presented with the same evidence of patterns which have more to do with where the police are concentrated than what people are actually doing. WOLF: So what do you say to the notion that these instruments are validated? That they predict? This information, whether there’s a potential bias incorporated into them, they still can predict six months to a year out whether someone is going to recommit a crime. MILLER: Yes, with great reliability. But it’s a population being normed against itself. And so, overwhelmingly concentrate criminal justice resources in a particular neighborhood, which leads to more arrests, which leads to more convictions, which leads to more imprisonment. Then I look at those who were imprisoned, and I use that to validate my measures. So the problem, is this sort of self-fulfilling prophecy, this feedback loop, this is one problem. Another problem is that, and Kelly Hannah-Moffat points this out brilliantly, correlation and causation are very different things. It’s like the standard social science response that any bench chair social scientist gives when they look at two relationships and people use that as some sort of cause, but likelihood that particular groups of people are more likely to commit a crime, doesn’t mean that having committed a crime in the past means you actually will commit a crime. And so what we’re doing is, we’re treating relationship as if it’s a cause, as if it’s a fact. And so I will sentence you now based on my assumption of your future danger to commit a crime based on a set of assumptions that I use to justify the overwhelming concentration of police to begin with. Police aren’t the culprits here. It’s a rationality, it’s a way to approach problems, that I think must be critically investigated. WOLF: And you also pointed out in your presentation that perhaps the cultural context, the environment and the changing policy culture where for instance, marijuana arrest which were so vigorously pursued several years ago are now considered a low priority, or they’re not even being done anymore. And yet, people have a record of those arrests and if history of arrest is a factor, someone in the audience also questioned this, should we drop those particular kind of...