New York Times
POSTED: 1:30 a.m. HST, Nov 2, 2012
In the tight 2004 campaign, the polls that asked Americans which candidate they supported — all the way up to the exit polls — told a confusing story about whether President George W. Bush or Sen. John Kerry would win.
But another kind of polling question, which received far less attention, produced a clearer result: Regardless of whom they supported, which candidate did people expect to win? Americans consistently, and correctly, said that they thought Bush would.
A version of that question has produced similarly telling results throughout much of modern polling history, according to a new academic study. Over the last 60 years, poll questions that asked people which candidate they expected to win have been a better guide to the outcome of the presidential race than questions asking people whom they planned to vote for, the study found.
Most recently, Mitt Romney won the Republican nomination despite at various points trailing other candidates — Rick Santorum, Newt Gingrich, Herman Cain, Rick Perry — in polls of Republican voters’ preferences. Even when Romney was behind, Republicans typically told pollsters that they expected him to win the nomination.
Asking about expectations, said Justin Wolfers, one of the study’s authors, “is fundamentally about treating poll respondents with respect.”
The question allows people to consider not only their views but also those of their relatives, friends and colleagues, he said. Some voters may also give more-honest answers about their own plans, rather than naming a candidate who briefly intrigues them, as happened in the Republican primaries this year. And an expectations question allows people to take into account speeches, debates and news media reports.
“More information produces better results,” said Wolfers, a professor of economics at the Gerald R. Ford School of Public Policy at the University of Michigan, who wrote the paper with David Rothschild, an economist at Microsoft Research, a division of the software company.
The paper — which the authors have presented at academic gatherings and will soon submit to journals — is generating discussion among public opinion experts. With response rates to polls having fallen sharply in recent years, thanks to mobile phones, caller identification and a rise in phone solicitation, expectations questions have the potential of effectively increasing a survey’s sample.
The authors estimate that a typical voter has as much information as about 10 randomly selected voters polled about their own intentions.
Frank Newport, editor-in-chief of Gallup, said he was intrigued enough by the paper to have talked with the authors about how to include expectations questions in more polls. “It’s treating every respondent like a mini-anthropologist,” he said.
Perhaps the biggest limitation of the study is that polling companies have asked expectations questions much less often than other kinds of questions, Newport added. “Like all intriguing hypotheses, we just need more data,” he said.
The study also offers another window on the current presidential contest and the endless debate over what the polls mean. In the last three weeks, polls — including by ABC/Washington Post, Gallup, Politico/George Washington University and Quinnipiac University/New York Times/CBS — have consistently found that more Americans expect President Barack Obama to win than expect Romney to win.
The margins havevaried between 13 and 24 percentage points for Obama amongsamples of likely voters, registered votersand all adults. The gap is similar to one from a Gallup poll in mid-October 2004, when 56 percent of respondents expected Bush to win, compared with 36 percent for Kerry.
The finding does not mean that Obama is sure to win. If anything, it highlights one uncertainty about the paper’s conclusion: Does it mostly reflect people’s knowledge of others’ voting intentions, or does it mostly reflect publicly available information?
Andrew Gelman, a statistics professor at Columbia University, said he thought the notion of asking people about their social networks was a smart one. But in this case, he said he believed that the finding largely reflected public information, which he considered less interesting.
Wolfers disagreed, saying he thought that voters’ predictions were based mostly on friends, yard signs and other private information, given that responses to the expectations questions varied so much.
Either way, expectations questions have the potential to be useful well beyond politics. If people turn out to be accurate forecasters as a group — be it because of their own knowledge or because of a wisdom-of-crowds synthesis of public information — they could help pollsters gain insight into the economy and consumer preferences.
Only 15 years ago, the response rate to a typical poll was about 40 percent. Today, it is only about 10 percent, said Andrew Kohut, the president of the Pew Research Center.
Rothschild said he believed that polling methods would need “to change dramatically” as “people make it harder and harder to contact someone randomly.” The beauty of an expectations poll, he and Wolfers said, is that it has the effect of turbocharging a poll’s sample.
In presidential races since 1952, the expectations question has pointed to the winner in 81 percent of states, based on data from the American National Election Studies. The question about voting intentions pointed to the winner in 69 percent.
A larger data set, including other U.S. elections and overseas elections, produced a similar result.
The study’s authors emphasized that expectations questions could produce useful results even with unrepresentative samples. On average, about 70 percent of people predict that their preferred candidate will win; if a poll only of Democrats found that 60 percent expected their candidate to win, that would suggest the Republican was the favorite.
Although expectations questions have not received much attention lately, Rothschild has discovered that they have a long history. In 1936, at the dawn of the modern polling era, The New York Times asked at least four political experts in each state to predict the outcome in their state.
Two well-known polling organizations at the time differed, with Literary Digest forecasting a victory by Alf Landon and the forerunner of the Gallup Poll predicting re-election for President Franklin D.
Roosevelt. The Times’ method suggested a Roosevelt landslide, and the article, written by Arthur Krock, appeared on Page 1 of the newspaper on Nov. 1, 1936.
“We hope,” Rothschild said, “that our paper can shift the focus of polling back to this amazingly powerful forecast mechanism.”