Why literature is the ultimate big-data challenge

By J.W.S.W.

NUMBER-crunching literary criticism was the butt of an academic in-joke in “Arcadia” (1993), Tom Stoppard’s cerebral play. Bernard Nightingale, a foppish poetry don, scoffs at a colleague who used a computer program to attribute an anonymous story to D.H. Lawrence. To Bernard’s “inexpressible joy”, he found that “on the same statistical basis, there was a ninety percent chance that Lawrence also wrote the ‘Just William’ books and much of the previous day’s Brighton and Hove Argus”. The “maths mob” skewered in Mr Stoppard’s play no longer seems so ridiculous; with the publication of the “New Oxford Shakespeare”, they have shaped the debate about authorship in Elizabethan England.

This new edition of the Complete Works made headlines last October as it identified 17 of Shakespeare’s 44 plays as collaborations (by comparison, the 1986 edition named only eight). The most thrilling new name on the contents page is that of Christopher Marlowe; his inclusion seems to give credence to authorship theories previously dismissed as conspiracies. What has really raised eyebrows, though, is the technique used to identify Marlowe’s hand: not traditional editorial insight, but computational analysis. So how do today’s data linguists figure out who wrote what, without confusing authorship and influence? And more importantly, why does it matter?

Computers and human readers can identify Shakespeare’s writing through “plus-words”—such as “gentle”, “answer”, “beseech”, “tonight”—which he uses frequently. This method becomes less accurate, though, when writers ape one another’s style as they often did in Elizabethan theatre-land. Early modern playwrights were a close-knit bunch and 16th-century audiences do not appear to have placed a high premium on novelty. “Tamburlaine”, Christopher Marlowe’s wildly popular play, spawned so many knock-off sequels and serials that Ben Jonson, a fellow playwright, felt compelled to lament the endless “Tamerlanes and Tamer-chams of the late age”. Shakespeare was as guilty of this as anyone. In “The Jew of Malta” (1589), Marlowe’s Barabas spies his daughter Abigail on a balcony:

“But stay! What star shines yonder in the east?

The lodestar of my life, if Abigail!”

If the lines sound familiar, it’s because Shakespeare’s Romeo echoed them ten years later:

“But soft! What light through yonder window breaks?

It is the East, and Juliet is the sun!”

With this mutual influence muddying the picture, how can computers tell the difference between Marlowe, Shakespeare, and Marlowe and Shakespeare drawing on one another? According to the editors of the “New Oxford”, the answer lies in “function words”. These are words like “to” or “a” that supply the grammatical mechanics of a sentence. The theory goes that all writers unconsciously use these words in distinctive ways. Shakespeare, for example, often put “and” next to “with”—Claudius marries Gertrude “With mirth in funeral and with dirge in marriage”, Old Hamlet’s ghost “Appears before them, and with solemn march / Goes slow and stately”. As a result, function words supposedly betray a writer’s identity, even when they’re trying to write like someone else. By analysing how a writer uses function words, computers can ostensibly identify their unique linguistic fingerprint.

In Shakespeare’s case, knowing who wrote what when matters because it helps to deflate the cult of Shakespearean exceptionalism, which imagines him as a freak of isolated genius. In the past, the main reason that bits of Shakespeare were attributed to co-writers was that they weren’t thought to be very good. Writers like George Peele and Robert Nashe, for example, were blamed for the leaden classical allusions that occasionally weigh down “Henry VI, Part I”. Similarly, acknowledging Thomas Middleton’s hand in a particularly choice section of “All’s Well That Ends Well” recognises that Shakespeare’s contemporaries often matched his skill. Even if the computer models aren’t infallible, the attempt to attribute work on the basis of evidence other than quality is welcome.

Secondly, correct attribution can inform our readings of the plays themselves. When the history plays are laid out in the First Folio, they look like a stand-alone project depicting a sequential sweep of history. Suggesting that most of “Henry VI, Part II” was written by Marlowe scuppers this vision of a unified body of work. Marlowe seems to be more interested in the role of ordinary people (like the lowly agitator Jack Cade) than he does in that of omnipotent monarchs; Shakespeare preferred the grand “stories of the death of kings”. By understanding that their authorship is messy, contested and symbiotic, we can better understand the plays themselves.

According to Gabriel Egan, one of the editors of the “New Oxford Shakespeare”, attribution models are becoming ever more accurate, partly because “computational people are increasingly turning to linguistic problems, because they are among the hardest problems that we can put computers to work on.” Where does this leave Bernard Nightingale’s fuming insistence that “you can’t stick Byron’s head in your laptop”? For now, he’s probably right. On a control test, even the advanced models used by the “New Oxford” sometimes misattributed works whose authorship we know for sure. For now, statistical analysis will remain one of many editorial tools. Nevertheless, it represents a strain of academic objectivity, rarely found in the field of Shakespeare studies. Surely that’s an idea that both Bernard and the maligned “maths mob” can endorse.