Mike Tipping has a column in Saturday’s Portland Press Herald criticizing conservatives and Maine reporters (specifically me and BDN politics writer Chris Cousins) for noting that a recent Public Policy Polling survey oversampled people who identified themselves as Democrats. 

The PPP survey showed that Democratic gubernatorial candidate Mike Michaud had a 7-point lead over Republican Gov. Paul LePage, while independent Eliot Cutler was in third place. The poll seemed like a bit of an outlier given that other surveys have shown the race as much closer.

Reporters aren’t typically polling experts, but we can read cross tabs, so I — and I’m assuming Cousins — went looking for something that could explain why PPP gave Michaud such a large lead.  

It wasn’t in the released demographic data, but there was a significant difference between the party affiliation sample, which showed that 40 percent of respondents identified themselves as Democrats, 32 percent said they were Republicans and 28 percent were independents.

The sampling contrasted with the most recent voter registration numbers from the Secretary of State’s Office: 37 percent of Maine’s more than 960,000 registered voters are unenrolled or independent. Nearly 32 percent identify themselves as Democrats while 27 percent identify themselves as Republicans. The rest are registered with the Green Independent Party.

I noted in the first write-thru of the story that a poll shouldn’t necessarily weight party ID to align with voter registration because party enthusiasm changes from year to year (For example, in 2010 a lot of independents registered as Republicans and a lot of Democrats stayed home, contributing to the GOP wave election). The qualifier was removed to right-size for the print edition, but even if it had remained in the story, it didn’t go far enough to put the party ID weighting in context.

Advertisement

In fact, highlighting the party ID sample in the first place was probably a mistake. 

Tipping wrongly assumed that I simply parroted the Republican spin of the poll and he seems to suggest that reporters should have been more credulous toward PPP because of the firm’s track record (They’re far from perfect and reporters should always be skeptical of polls.). But his column and Pew Research do explain why focusing too much on a poll’s party ID sample is a bad idea. 

Pew notes right off the top that "focusing on the partisan balance of surveys is, in almost every circumstance, the wrong place to look" when looking for bias or flaws.

"While all of our surveys are statistically adjusted to represent the proper proportion of Americans in different regions of the country; younger and older Americans; whites, African Americans and Hispanics; and even the correct share of adults who rely on cell phones as opposed to landline phones, these are all known, and relatively stable, characteristics of the population that can be verified off of U.S. Census Bureau data or other high quality government data sources. … Party identification is another thing entirely. Most fundamentally, it is an attitude, not a demographic. To put it simply, party identification is one of the aspects of public opinion that our surveys are trying to measure, not something that we know ahead of time like the share of adults who are African American, female, or who live in the South." 

And this:

"In short, because party identification is so tightly intertwined with candidate preferences, any effort to constrain or affix the partisan balance of a survey would certainly smooth out any peaks and valleys in our survey trends, but would also lead us to miss more fundamental changes in the electorate that may be occurring. In effect, standardizing, smoothing, or otherwise tinkering with the balance of party identification in a survey is tantamount to saying we know how well each candidate is doing before the survey is conducted."

In other words, there could be a lot of reasons why the PPP poll seems like an outlier, but its party ID sample is highly unlikely to explain it. 


Only subscribers are eligible to post comments. Please subscribe or login first for digital access. Here’s why.

Use the form below to reset your password. When you've submitted your account email, we will send an email with a reset code.