The algorithms that underlie the Facebook news feed and Google search results are proprietary and ever-evolving. Christian Sandvig studies how the details of such algorithmic operations are selectively disclosed to shape the public’s relationship to these services. Some people are blissfully ignorant, some aware enough to be indignant or awestruck, but very few are fully informed. He emphasizes a very simple point that has been relevant since the early days of commercial computing systems: algorithms are created with the inherent bias of their creators. Now known as Crandall’s Law of Algorithms (Named after Robert Crandall, legendary airline executive and former CEO of AA), this point is something Wall Street, Facebook, and Google understand perfectly well—it’s fundamental to their business models—and should similarly inform our own understanding of our relationships to these services.
DateOctober 16, 2014TitleBenjamen Walker’s Theory of Everything: Enchanting By NumbersSegment10:47 - 14:33Christian Sandvig:
If you like computer history, you’re familiar with the Saber system. It’s sometimes described as the first wide-scale commercial application of computing. At the time that it was turned on, it was the largest commercial computer network in existence in the world, and it did airline reservations. It was a system that allowed ticket agents and travel agents to search for flights and buy tickets with a special computer terminal that was provided by Saber. The funny thing about it is that the user of the system started to notice that many of their requests resulted in improbable itineraries. So the system might recommend an extremely long and expensive flight as a top result, and that seemed odd. Well, it turned out that since American Airlines built the reservation system, they had a unit at American that was informally referred to as the Screen Science Unit. And that unit’s job was to think about how they might sort the results in the airline reservation system so that American would make more money. This was something of a scandal at the time but when the CEO of American was called before congress to testify he didn’t understand why people were upset. He said of course American is using the system to make money… why would we build an airline reservation system if we weren’t going to use it to advantage our own airline?
(…)
We should expect this behaviour from all of our algorithms. Why wouldn’t an algorithm be designed to advantage the company that invested millions of dollars in building it? The things we have to watch out for are the places where what benefits the company doesn’t benefit us.