Online polls are rising, and so are concerns about their results

Two of the most important trend lines in political campaign polling might be about to cross.

One is the rise of online polling. The other is the decline of telephone surveys because of rising costs and possibly declining quality, tied to the fact that fewer Americans are willing to participate in a telephone poll. Many public opinion researchers have thought that Internet polling would eventually overtake phone polling, but they hoped it would not happen before online polling was ready.

Ready or not, online polling has arrived. Political analysts and casual poll readers now face a deluge of data from new firms employing new, promising, but not always proven methodologies. Nowhere is the question of the accuracy of the new online polls more evident than in the survey results for Donald Trump. He fares better in online polls than in traditional polls, and it’s not clear which method is capturing the public’s true opinions.

According to the Pollster database of Republican primary polls, there have been nearly as many Internet surveys as there have been traditional, live-interview telephone surveys, 90 versus 96. By this time in 2011, opinion researchers had conducted just 26 online surveys and more than 100 live-interview polls.

Well-known companies, as well as smaller startups, are doing online polling. Every day, Reuters/Ipsos publishes a new iteration of its online tracking poll — the only tracking poll, telephone or Internet, of this election cycle. Morning Consult, an organization that didn’t even exist in 2012, now publishes a weekly poll. SurveyMonkey, despite its seemingly unserious name, has conducted election polls for NBC News, The Wall Street Journal and The Los Angeles Times. The Wall Street Journal sponsored a post-debate survey from Google Consumer Surveys. CBS News conducts surveys with YouGov, an Internet market research firm based in Britain.

The abundance of Internet-based polling reflects the extent that it has become easier over the past decade. According to Pew Research, 87 percent of Americans now use the Internet, so an online survey can cover most of the population.

But big challenges remain. Random sampling is at the heart of scientific polling, and there’s no way to randomly contact people on the Internet in the same way that telephone polls can randomly dial telephone numbers. Internet pollsters obtain their samples through other means, without the theoretical benefits of random sampling. To compensate, they sometimes rely on extensive statistical modeling.

The online pollsters may get clumped together because they use the Internet, but there’s far greater diversity of methods among these polls than among telephone polls. The diversity is a reflection of the still-developing science of Internet polling. No one is sure of the right way to do it.

Reuters has the most traditional approach: It recruited its panel from a traditional telephone and mail survey. YouGov and Morning Consult use panels recruited from a variety of sources on the Internet. Google entices people to take a poll. SurveyMonkey has the most novel approach: It turns to the millions of people who participate in any number of SurveyMonkey’s unscientific polls on other subjects — the kind you can make yourself — and adds questions about politics.

The statistical techniques that the Internet pollsters then use to adjust these data vary nearly as much.

But are the results the same as they would be through traditional polling? Already, it is becoming clear that online polls favor one candidate. Ever since Trump rose in the polls, he has fared best in the online ones — sometimes by as much as 10 points better than live-interview telephone surveys conducted over the same period.

There are a number of possible explanations for Trump’s strength in online polling, which was first noted by Jonathan Robinson, an analyst for Catalist, a data firm associated with the Democratic Party.

One is that voters are likelier to acknowledge their support for Trump in an anonymous online survey than in an interview with a real person. Plenty of research suggests that the social acceptability of an opinion shapes the willingness of poll respondents to divulge it, and it’s imaginable that voters would be reluctant to acknowledge support for a controversial figure like Trump.

The data is consistent with this notion, even if it is not sufficient to prove it. Trump is the only candidate who does appreciably better or worse in online than live interview surveys, and automated phone surveys using interactive voice response technology rather than live interviews also tend to show Trump faring better.

If Trump is faring better in the online polls because of social acceptability bias, then the online polls might be even more accurate than the traditional surveys. But we don’t know. What is also possible is that the online polls — without the benefit of random sampling — have biased and unrepresentative samples.

The fact that online pollsters have sometimes relied on extensive modeling to adjust their results could be a sign of the limitations of online polling. The challenge is even worse during primary season, when the statistical tools used in the general election don’t work nearly as well.

Some online pollsters, for instance, adjust their sample to match the ideology and party identification of voters, which are two extremely strong predictors of how people will vote in a general election. If you can nail the balance of Democrats and Republicans, liberals and conservatives, whites and nonwhites, you can probably get the right result. A group of political scientists used this exact technique to poll the 2012 presidential election using data collected from Xbox.

But this heavy-handed technique isn’t as useful in a primary. Nailing the right number of Democrats and Republicans may help a general election poll, but there’s no similar question that helps make sure a poll has the right number of voters for Trump. If weighting by partisanship is necessary in a general election, one wonders how much quality declines in a primary, without the same crutch.

The new Internet approaches represent high-profile, very public experiments in the next generation of opinion research. The problem is that we don’t yet know which ones will pan out, even if Trump knows now exactly which polls he should tweet about.