Earlier this year, Apple launched a credit card with plenty of fanfare. The spending power of a credit card plus the technological savvy of Apple — what could go wrong? Plenty, it turns out, capped by numerous accusations of sexism. Turns out that women were offered credit limits much lower than the ones men were offered — an alarming development that’s prompted Apple to allow card applicants to appeal said limits.
But the question remains: how did an algorithm — which should, in theory, not be sexist — let something like this happen?
A new article at Quartz by Dan Kopf offers some insight into exactly that topic. And it turns out to be connected to Goldman Sachs’s World Cup predictions to boot. (Feel free to enter your own “red card” or VAR-related joke here.) “There is no evidence yet that the algorithm is sexist, beyond these anecdotes,” Kopf writes. “But a lack of transparency has been a recurring theme.”
When Goldman Sachs is silent on what makes their algorithm tick, it can be instructive to look at some of their past algorithms for cues on this one. Kopf explores how Goldman Sachs used machine learning to pick the winner of the 2018 World Cup. “The problem with using a machine learning method is that it makes it hard to explain how a prediction works,” he writes.
Goldman Sachs’s 2018 World Cup predictions, you may recall, offered a vision of Brazil defeating Germany in the final. It didn’t exactly go down like that, to the frustration of Brazilian soccer supporters everywhere. As Kopf writes, this model may not have been the best one for this task: “A model that was at least easier to explain may have been more useful,” he writes.
Is this what went wrong with the Apple Card’s credit limits? It’s certainly a possibility. And we’re probably not far from a moment in time when machine learning will help scientists predict whether machine learning was used in a given situation. Only machine learning knows for sure.
Subscribe here for our free daily newsletter.
Thanks for reading InsideHook. Sign up for our daily newsletter and be in the know.
