For several weeks this summer, in certain cars of
the New York City subway, Spotify, the music-streaming service, bought
out all the ad space to promote one of its newer features: Discover
Weekly. The service sends users a personalized playlist of music every
Monday.
New-music discovery is one of the fiercest battlegrounds on which
competing music-streaming services fight, and as they do, they have a
tactile decision to make: should humans or computers take the lead?
In other words, will that delightful new song that pops up in your
playlist do so primarily because a music expert, after considering beat
and vocals and instrumentation, decided you’d like it, or because an
algorithm cycling through vast bits of data, personal and global,
arrived at the conclusion that you would.
If the song’s good, it probably doesn’t matter by which method it was
selected. But what was interesting about Spotify’s ad campaign is how
much it emphasized its use of algorithms, presenting them even as less
impersonal than, perhaps, some nameless song analysts in a snazzy
tech-startup office somewhere.
“The Discover Weekly playlist on Spotify really is like unwrapping a
birthday present every Monday. Algorithms, you get me like no other,”
read one Tweet that later appeared in a subway ad.
“It’s scary how well @Spotify Discover Weekly playlists know me. Like
former-lover-who-lived-through-a-near-death experience-with-me well,”
crooned another.
Apparently in the age of Big Data, our relationships with algorithms can be more intimate than our relationships with people.
If this is creepy — and it is creepy — that’s because intimacy was
the exact thing computers were supposed to remove from our daily
interactions. After all, another promise of the Discover Weekly service
is that it works equally for everyone, unlike its predecessor, the
record-store clerk, who blew past as you flipped through Nickleback to
suggest the next hippest artist to the cute girl browsing the selection
of Nirvana.
Music is a special case. It’s a place where we want bias. We want the
record-store clerk to linger by us a bit longer because he likes us and
our taste more than he does that of the other shoppers. Perhaps this is
why we so eagerly hope that the algorithms that select our music favor
us, even when we know algorithms aren’t supposed to favor anyone.
In her new book
Weapons of Math Destruction, Cathy O’Neil
argues we should take the conclusions of many other algorithms just as
personally. Bias, she laments, has tainted a whole host of other
algorithms, ones we rely on to make decisions for us that we expect
— that we assume — will be fair to everyone evaluated. In a wide-range
of realms including criminal justice, job evaluation and hiring, and
political and product messaging, we count on algorithms to make
judgements that for decades were made by people whose whims influenced
outcomes. The promise is that the algorithms do this fairly.
A judge may be acculturated to believe that black males are more
likely to reoffend, but when an algorithm suggests the sentence he
gives, gone are the charges of race being a factor. A teacher may be the
principal’s poker buddy, but when his students’ test scores are
crunched by a computer and out comes a dismal rating of his ability,
it’s time for him to go.
"The problem is that so many algorithms that dictate our lives are often based on junk assumptions."TWEET THIS QUOTE
In the age of algorithms all a decider has got to do is feed in the
raw data and wait for a number to pop out. Gone is the trying
deliberation of casting judgement, of weighing each of those variables,
calling on a limited set of prior experience, and trying to push emotion
aside. An algorithm knows everything, feels nothing, and never wakes up
on the wrong side of the bed. The decider doesn’t even have to know how
the thing works. That’s the era when magic is spelled STEM.
So why then do so many of the algorithms’ outcomes that O’Neil
documents seem to recreate the racial biases and arbitrary
decision-making of that earlier, personal era? Why are our sentencing
algorithms treating black males more harshly and firing prized teachers?
Her answer isn’t mind-blowing: algorithms are powered by math, not
magic. The equations behind them must be constructed by people and the
inputs must be measured and transformed into numbers, again in a process
dictated by people. The biases of our culture are recreated in the
system. Neither surprising is the construction of her book to readers of
the popular-science genre: it’s a series of case studies that all
reduce to the bite-size conclusion of whether or not the particular case
represents a Weapon of Math Destruction, or WMD (a groaner, indeed, but
at least we all know what she means).
What’s fierce about O’Neil’s book is her authority and how
specifically she can diagnose the problem and proclaim a bold solution.
Far from a yesteryear nostalgist, O’Neil, who earned her professional
credentials working in the very tech-mathematics combine she criticizes,
doesn’t see all algorithms as pernicious. The problem, she finds, is
that so many algorithms that dictate our lives are often based on junk
assumptions, operate in the dark with no lay understanding (and often
little professional understanding) of their mechanics, and unlike the
racist judges and nepotistic functionaries of the past, don’t have
anyone to complain to. Moreover, algorithms often lack self-correcting
feedback mechanisms, instead their errors are treated as the cost of
doing business and because we’ve fallen for the seeming magic of the
algorithms they are rarely interrogated as they should be.
More worrisome, O’Neil shows that it’s becoming increasingly more
difficult to determine when an algorithm may be giving objectionable
results — that I, adjudged a good credit risk, may only be seeing
internet ads for low-interest auto loans, while you, determined to be a
poor one, are bombarded only with ones of usurious payday loans, without
either of us knowing what’s on the other’s screen. Call this the
pandering politician problem: if a candidate tells each voter exactly
what they want to hear — a product political firms wielding algorithms
are ever working to hone — and is able to do it in private, who’s to
know when they’ve broken their campaign promises? Who’s to stop them
offering each of us personally tailored lies?
The answer seems to be no one, and it’s here, when offering a
solution, that O’Neil turns most to the past. Looking back at how the
nation confronted the ills of previous technological change, O’Neil
returns with a simple answer: regulation. In neoliberal Silicon Valley
such a proposal may be heretical, but you can’t STEM your way out of
everything.
Until then, perhaps we need a new subway ad: “Algorithms, you get me like no other, even though I don’t get you.”
