The data of the dark web

By THE DATA TEAM

SINCE the launch of the Silk Road five years ago, dark-web markets have represented a shadowy and much-maligned corner of the internet. And the secretive nature of such sites makes them difficult to study. But last year a researcher using the pseudonym Gwern Branwen cast some light on them. Roughly once a week between December 2013 and July 2015, programmes he had written crawled 90-odd cryptomarkets, archiving a snapshot of each page.

The Economist has extracted data from the resulting 1.5 terabytes of information for around 360,000 sales on Agora, Evolution and Silk Road 2. There are, inevitably, flaws in the data. Mr Branwen’s scrapes probably missed some deals. We excluded any sale that was more than a week old when the scrape took place. If a price was absurdly high we ignored the page; such “holding prices” are used by dealers to indicate a lack of supply. Vendors may fake sales (though probably not often, since cryptomarkets take a cut) or reviews (though dissatisfied real customers would soon catch outright fraudsters). The volatile exchange rate between bitcoin and dollars means our conversions of prices are not completely accurate. Yet the data still reveal a lot about the workings of a once-shadowy industry.

In total the deals were worth around $50m. Of those MDMA (ecstasy) sold the most by value while marijuana was the most popular single product, with around 38,000 sales. Legal drugs such as oxycodone and diazepam (Valium) were also popular. A third of sales did not belong in any of our categories: these included drug kit such as bongs, and drugs described in ways that buyers presumably understood, but we did not (Barney’s Farm; Pink Panther; Gorilla Glue).

Read our full analysis of dark-web markets, the price of online drugs and how competition is changing the narcotics industry here.