Case Study for Social Responsibility in Computer Science

You are a data scientist working for FutureCrime Inc., a company that provides “risk assessment” software for use in criminal justice, especially sentencing and parole decisions (not determination of guilt). The software takes into account a wide variety of offender characteristics, and compares those characteristics with a database of known offenders. Based on that comparison, the FutureCrime software computes a “recidivism risk score”, indicating the likelihood that the offender will re-offend in the near future. The score is based not only on what the defendant has done, but on the recidivism rate of those who resemble the defendant. A judge can use this score when setting the length of a prison term, or making a parole decision. One day a large new dataset is presented to you for incorporation into the database. Upon inspection, you realize that the new data is heavily biased due to the manner in which it was collected. Put simply, the data suffers from the same “filter bubble” problem that FaceBook has. An algorithm dispatches police patrol cars to high crime neighborhoods, which increases the number of arrests made in those neighborhoods, so more cars are dispatched… In this particular case, the initial definition of “high crime neighborhood” was determined by police who liked to patrol particular streets due to personal bias. It is clear that including this data will gravely disadvantage offenders about to be sentenced who resemble those in the new dataset. ************* Some of you will recognize this story. Northpointe Inc., developed the COMPAS system, described by the developers here . COMPAS was the subject of an article by ProPublica , which caused considerable controversy. Northpointe posted a rebuttal , which prompted yet another rebuttal . The Washington Post has a good summary here . For a more recent, and cool-headedly academic perspective, see here . ************** So what are we writing about? Section 1 of the Case Study Template asks for a summary of the facts of the issue. This is where you can make clear whether you are concentrating solely on FutureCrime Inc. (which is totally fine), or pulling in additional issues from Northpointe. It’s a deep dive. With the facts established, you can take ethical and legal positions as you see fit.