1/ There has never been a more concentrated distillation of my teaching than this lesson: Algos, Bias, Due Process, & You. It is the apotheosis of what I do. I very much hope you enjoy it, share it, and make bits of it your own. https://suffolklitlab.org/algos-bias-due-process-you/
Discussion
Loading...
2/ TL;DR: I built a bunch of highly-modular online simulations you can use with your students. They cover automation bias,¹ the false positive paradox,² competing definitions of fairness,³ disparate impact resulting from machine bias,⁴ and the value of due process.⁵
¹ https://www.davidcolarusso.com/42/citation_assistant/
² https://bail-risk-simulator-50382557550.us-west1.run.app/
³ https://fairness-simulator-the-toilet-seat-dilemma-50382557550.us-west1.run.app/
⁴ https://facial-recognition-bias-sim-50382557550.us-west1.run.app/
⁵ https://screening-vs-diagnostic-tests-50382557550.us-west1.run.app/
4 media
3/ Most of them are self-explanatory. However, the first one intentionally lacks in-simulation context, and I think linking them together creates a more-compelling narrative arc. Play with the links in isolation (found in the above & in the blog post), or read the post for the guided tour. https://suffolklitlab.org/algos-bias-due-process-you/
4/ The seed of the class was this reporting from 2016, "Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks." In it, the authors describe the use of a risk assessment tool by courts deciding questions like the granting of bail. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
5/ When that article was published I was a data scientist at the public defenders, and in my corner of the world, it created a bit of a furor, kicking up discussion around a set of issues I wanted students to understand. I thought, maybe they could role play as the court in a similar setup. So, I asked myself what students needed to understand to have an informed discussion, recognizing that I only had an hour and 50 minutes with them. 🤔
6/ Here's what I came up with:
- accuracy isn't always the right performance measure
- mathematical models encode and replicate the biases found in their training data
- there can be competing and contradictory ideas of what makes something fair
- under certain conditions people are likely to over-rely on machine outputs (automation bias)
- the choices we make about how to use tools embody and reveal what we value
7/ It occurred to me that all they knew about my session was that it would be on “algorithmic bias.” What if I could get them to experience automation bias first-hand? That would make it harder to dismiss as something that only other people fall victim to. . . . A plan began to form.
1+ more replies (not shown)
4/ The seed of the class was this reporting from 2016, "Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks." In it, the authors describe the use of a risk assessment tool by courts deciding questions like the granting of bail. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing