When weapons can think for themselves

Army of None: Autonomous Weapons and the Future of War. By Paul Scharre. W.W. Norton & Company; 448 pages; $27.95.

ARTIFICIAL intelligence (AI) is on the march, for good and ill. The AI that makes possible self-driving cars and diagnoses diseases more accurately than doctors will save lives. The AI that does jobs better than workers may be a more mixed blessing. But AI that might give machines—“killer robots”—the responsibility for deciding how wars are fought, and who gets killed, is a science-fiction nightmare. Paul Scharre, a former army ranger, explores this dystopian prospect in “Army of None”.

Mr Scharre interviews the engineers building autonomous weapons and the strategists preparing for their arrival. He sees first-hand what to expect from these technologies, such as swarms of small, low-cost drones locked in aerial combat, manoeuvring with superhuman co-ordination. The speed of these developments, he finds, both excites and disturbs the military establishment. The generals know they are entering an era in which algorithms will determine success on the battlefield, and humans may be unable to keep up with the pace of combat.

A key distinction, Mr Scharre says, is between new weapons that are equipped with fairly “narrow” autonomy, and will carry out a specific task more effectively than humans, such as an anti-ship missile that has been programmed to select its targets; and future systems (perhaps 20 years away) endowed with “general” AI. Weapons with narrow autonomy that are able to activate themselves—for example to respond to a cyber-attack at the speed of light—will pose some risks of escalation. But when combined with human-controlled systems, as in a squadron of drones orchestrated by a manned aircraft, they may improve precision and situational awareness, and be capable of better split-second analysis than people, who are prone to fear, rage and exhaustion.

Fully autonomous weapons which can plan, solve problems and extrapolate from experience are something else; they will need a human only to order the start of a mission (maybe not even that). Here Mr Scharre thinks the dangers and moral issues are so profound that it is in humanity’s interest to seek ways of controlling the technology. Unlike campaigners against “killer robots”, he does not believe they can simply be banned. Instead, he argues for ensuring a minimum degree of human involvement in their operation and compliance with international law.

That sounds comforting. But arms-control agreements work only when there is reliable verification. The essence of autonomy, writes Mr Scharre, is software rather than hardware, making transparency very difficult. Liberal democracies may insist on “meaningful” human oversight, but will every country be as fastidious? And given the ubiquity of AI, what use might terrorists, devoid of compunction, make of it?