September 30, 2012

Comparing Polls: Varied results arise from differences in methods

Findings from four surveys differ in part because of live interviews and including cellphone users.

By John Richardson jrichardson@pressherald.com
Staff Writer

(Continued from page 1)

Charles Franklin

Related headlines

ABOUT THE POLLS

HERE IS a quick look at the four recent Maine polls.

CRITICAL INSIGHTS: Sept. 12-16, live interviews with people on landlines and cellphones, 618 likely voters with a 4 percent margin of error.

MAINE PEOPLE’S RESOURCE CENTER: Sept. 15-17, automated phone calls, landlines only, 856 voters in likely voter households with a 3.35 percent margin of error.

PUBLIC POLICY POLLING: Sept. 17-18, automated phone calls, landlines only, 804 likely voters with a 3.5 percent margin of error.

RASMUSSEN REPORTS: Sept. 25, automated phone calls on landlines only and some Internet-based surveys, 500 likely voters with a 4.5 percent margin of error.

POLL VARIATIONS

Charles Franklin is the founder of PollsAndVotes.com, a website devoted to nonpartisan analysis of the details of polling and election outcomes, mostly in the U.S. He also is a University of Wisconsin at Madison political science professor and the director of the Marquette University Law School Poll.

Franklin is no stranger to poll variation and controversy.

His polls before the June 5 gubernatorial recall vote in Wisconsin led Democrats and bloggers to accuse him of having Republican ties and favoring Republican Gov. Scott Walker. While most polls had the recall effort much closer, Franklin's final poll showed that Walker was winning the recall election by 7 percentage points. Walker survived by 6.8 points.

Last week, Franklin said, he was under fire from Republicans because of his latest presidential poll results.

Cellphones also may have contributed to stronger support for the same-sex marriage referendum in the Critical Insights poll. Young voters, the ones most likely to have cellphones and no landlines, are strongly behind the legalization effort.

The use of live interviewers instead of automated voice recordings might also have contributed to more same-sex marriage support in the Critical Insights poll.

Pollsters have clearly found that voters are more likely to say they support gay marriage than to actually vote that way in the privacy of a voting booth. It may be that voters opposed to same-sex marriage are more willing to say so with computerized calls than live interviewers, Franklin said.

"I think that there could be some effect here." But, he said, "the bottom line for me is that you have three polls here that are all pointing in the same direction."

One other difference is worth considering, he said.

Critical Insights is the only one of the three polls Franklin reviewed that mathematically adjusted the results to match the party affiliations of Maine's voting population.

While pollsters all tend to "weight" their samples by age and gender, they are split on whether to statistically adjust by party affiliation.

The polls by Public Policy Polling and Maine People's Resource Center did not weight by party, although the surveys included a higher percentage of Democrats and a lower percentage of independent voters than the state population in general.

Both methods are valid and neither is perfect, Franklin said. "Different pollsters might make good principled arguments both ways."

Party affiliations may have had some effect on the polls, but it doesn't appear to be a large one.

Critical Insights' unweighted data shows its adjustment for party affiliation slightly increased King's lead but had almost no effect on the same-sex marriage question.

Meanwhile, the Rasmussen Reports poll released Thursday also was weighted by party affiliation, and its results were closely in line with the polls by Maine People's Resource Center and Public Policy Polling.

In the end, Franklin said, all four polls are estimates, and they're telling the same story.

"We certainly shouldn't expect strong agreement on all of these numbers," he said, "but my sense is they line up pretty good."

Publishing poll results two weeks after a survey is conducted -- as the Press Herald did today -- is unusual for horse-race polls and potentially risky because events could change the race dynamics in the meantime, Franklin said. It also opens a poll to more criticism than usual from campaigns that don't like the numbers, he said.

But such a delay in publishing results is not unethical if disclosed, and it is less unusual in the case of value-added polls used to explore issues and trends, Franklin said. 

Staff Writer John Richardson can be contacted at 791-6324 or at:

jrichardson@pressherald.com

Were you interviewed for this story? If so, please fill out our accuracy form

Send question/comment to the editors




Further Discussion

Here at PressHerald.com we value our readers and are committed to growing our community by encouraging you to add to the discussion. To ensure conscientious dialogue we have implemented a strict no-bullying policy. To participate, you must follow our Terms of Use.

Questions about the article? Add them below and we’ll try to answer them or do a follow-up post as soon as we can. Technical problems? Email them to us with an exact description of the problem. Make sure to include:
  • Type of computer or mobile device your are using
  • Exact operating system and browser you are viewing the site on (TIP: You can easily determine your operating system here.)