There are dozens of cognitive biases that influence our thoughts and behaviors, and for a while, there was a belief that automated systems were the only way to really get clear and balanced decisions. Of course, others would argue that such systems are far from bias-free and developer bias is often hard-coded in.
A recent paper explored the gap between how we intend to use algorithms and how we actually use them. The study examined their application in both digital journalism and criminal justice. It emerged that workers in both fields use remarkably similar strategies to deal with algorithms.
"Whereas managers and executives frequently emphasize how 'data-driven,' modern, and rational their organization is, the actual uses of algorithmic techniques in web newsrooms and criminal courts reveal a more complicated picture," the authors say.
Bypassing the Digital Support
For instance, workers in both environments were given digital tools to support their work, whether it's real-time analytics platforms for the journalists or predictive risk analysis tools in court rooms. In both, however, workers would often opt out of using the tools that were supposed to be there to help them.
For instance, in newsrooms, many journalists would deliberately avoid using the analytics platform, with many revealing that writing for pageviews often resulted in poorer-quality stories.
A similar picture emerged in courtrooms, where algorithms were introduced to try and overcome apparent racial biases in the system, but they remain seldom used by judges due to the perceived bias they themselves (might) have.
"I'd prefer to look at actual behaviors," one judge told the researchers. "With these tools, the output is only as good as the input. And the input is controversial."
Central to the concerns both sets of workers have is the reliability of the algorithms they're given. Many of the current generation are opaque and not at all transparent; therefore, workers have little idea as to their workings.
There are attempts to make algorithms capable of explaining their workings more effectively. For instance, earlier this year, researchers developed an algorithm that is not only capable of performing its task but also translates how it achieved it into reasonably understandable English via a documentation process that is performed at each stage of its work.
Suffice to say, its capability to do that is rather limited thus far, as it's only capable of describing its work in recognizing human behavior in pictures.
The algorithm trains itself using two distinct data sets. The first is to help it figure out what's going on in the photo, and the second is to help it answer how it did so. The first half of this task is fairly standard, with a series of labeled images fed the algorithm. The second half, however, is quite novel in that it utilizes three questions alongside each image, with 10 possible answers to each of them. So it might ask, "Is the person cycling? No, because... the woman doesn't have a bicycle."
This then gives it a degree of context around how it came to identify what was in the picture. It's something the research team refer to as a "pointing and justification" system in the sense of being able to justify any data that you care to point at.
It's fairly a modest beginning but hints at a future with greater levels of transparency, which may in turn help to build the kind of trust in digital systems that seems to be lacking today.
"People who design these tools do not always follow closely how they are being used in specific organizations," the authors conclude. "Sometimes people use technology in ways you want them, but sometimes they use it differently."