The human inside the machine

By Tom Standage

In 1770 a chess-playing robot, built by a Hungarian inventor, caused a sensation across Europe. It took the form of a life-size wooden figure, dressed in Turkish clothing, seated behind a cabinet which had a chessboard on top. Its clockwork arm could reach out and move pieces on the board. The Mechanical Turk was capable of beating even the best players at chess. Built to amuse the empress Maria Theresa, its fame spread far beyond Vienna, and visitors to her court insisted on seeing it.

The Turk toured Europe in the 1780s, prompting much speculation about how it worked, and whether a machine could really think: the Industrial Revolution was just getting started, and many people were questioning to what extent machines could replace people. Nobody ever quite guessed the Turk’s secret. But it eventually transpired that there was a human chess player cleverly concealed in its innards. The apparently intelligent machine depended on a person hidden inside.

It turns out that something very similar is happening today. Just like the Turk, modern artificial-intelligence (AI) systems rely on help from unseen humans. Training a “deep learning” system involves showing it millions of examples of an input (such as photographs of animals) and telling it what the correct output is (“cat”, “dog”) in each case. Once trained, the system can then correctly identify animals in pictures it has not seen before. But the training process requires huge numbers of correctly labelled examples – and those must be created by humans.

As a result, an entire industry has sprung up, in which armies of human workers provide the data needed to train AIs. Mighty AI, for example, based in Seattle, has an online community of 300,000 people who carefully label images of street scenes to train self-driving cars. “We want cars to have human judgment,” its boss told me, “and for that we need human expertise.” The big tech firms have their own private “crowds” of online workers who do similar work.

The labelling work can also be farmed out to an online “crowdworker” platform. The largest of these platforms, where people are paid to perform simple tasks that are beyond the capabilities of computers, is called Mechanical Turk. This is a direct reference to the fact that, just as with the 18th-century original, its workers provide the human expertise that powers supposedly intelligent systems.

Without realising it, you are helping out with these tasks too. When you log onto something online, you may be asked to transcribe some distorted text, or identify the images within a grid that contains street signs or vehicles. This kind of task, known as a CAPTCHA, is used to verify that you really are a human, and not a bot – because computers still cannot perform such tasks reliably. The results of these tests, millions of which are completed each day, are used by Google and others to transcribe old books, label images for self-driving vehicles or to improve mapping services.

Your online activity helps to train online systems in other ways, too. When you tag a photo on Instagram with #cat or #dog, such labels allow your images to be used for training. Google keeps track of which search results people click on, and uses that information to determine the order in which search results will be shown to other users in the future. Spotify uses the playlists created by its users to help it recommend music tracks to others. Gmail suggests replies to emails based on analysis of millions of its users’ previous replies.

Pretty much everything you do online creates a trail of data that can be used for making systems smarter. As Google, Facebook and others operate their enormous smart machines, we are all helping to power them. A clockwork chess robot from the 1770s thus foreshadowed both the modern debate about artificial intelligence – and a key aspect of making the technology work. The internet is a giant Mechanical Turk: whether we know it or not, we have all become the people inside the machine.■

Image: Getty