The Economist’s analysis of polls taken during Mr Trump’s impeachment proceedings affirms Ms Kennedy’s suspicion. In polls that weight their data to represent America’s partisan balance or the results of the 2016 election, the share of adults who approve of Mr Trump’s job as president has risen by half a percentage point since impeachment proceedings began in earnest last October. But in polls that do not, Mr Trump’s ratings have increased by over three percentage points.
Data from YouGov, which conducts online polls with The Economist and adjusts its numbers for partisan bias, confirm that Republicans have become more likely to take part in their surveys over the past four months. In early September, before Nancy Pelosi announced impeachment proceedings, YouGov had to weight the attitudes of Republicans about 10% higher than the average respondent in order to achieve a sample that accurately reflected the results of the 2016 election. This higher-than-average weight implied that Republicans were less likely than Democrats to take their surveys. But by the first week of February, the firm’s weight for the average Republican was the same as for the average respondent, suggesting that they had become much more likely to fill out their surveys.
Hence pollsters who do not adjust for partisan bias have picked up a recent—and mostly phantom—swing in Mr Trump’s favour. Yet Ms Kennedy is quick to say that weighting is not a magic wand for ensuring high-quality results. A recent Pew report found that online surveys, in particular, can also be unduly influenced by so-called bogus participants who could be causing even more errors in pollsters’ measurements by submitting nonsensical, and disproportionately positive, responses. “Although it is not a large effect,” says Ms Kennedy, “it’s systematic”, and could be enough to nudge even a collection of different polls in one direction. Such are the challenges pollsters face if they want to be sure that movements in public opinion—like the recent shift towards Mr Trump—are in fact real.■
Dig deeper:
Sign up and listen to Checks and Balance, our new newsletter and podcast on American politics
