Eleven Three Twenty: How to Read Political Polls Like a Pro
October 5, 2020
Election Day is now only four weeks from tomorrow, which means we are about to be flooded by a tsunami of daily polls. Some of these polls will be high-quality snapshots of where the election stands. Others will be absolutely useless due to the pollster’s bias or poor methodology. Still others may be designed well but still be a statistical outlier — an apparently random result that is not an accurate reflection of the current state of the race. (Even well-designed polls by historically reputable pollsters sometimes generate outlying results for no discernible reasons.)
To help you more accurately understand and assess the polling numbers with which you’ll soon be bombarded, I thought I’d share five questions you should ALWAYS ask yourself when reading poll results.
Question 1: What organization conducted this poll?
Not all pollsters are created equal. Some have a solid track record of designing and implementing high-quality scientific political polls. Others are thinly discussed public relations or marketing companies who administer non-scientific “polls” for their clients (either media outlets or the campaigns themselves).
If you’re unsure about the reputation of a pollster, check out FiveThirtyEight.com’s “Pollster Ratings.” It’s a detailed, unbiased grading system of the various pollsters.
For example, the following five often-cited pollsters have received “A+”, “A”, or “A-” grades from FiveThirtyEight:
- ABC News/The Washington Post
- Monmouth University
- Siena College/The New York Times Upshot
- Fox News/Beacon Research/Shaw & Co. Research
Conversely, some very recognizable names (SurveyMonkey, Google Surveys, and Brown University, to name but a few) have some variation of “D” grades.
Question 2: Who is being interviewed and how were they interviewed?
Okay, okay, this is really two questions. But they’re closely related. First, the “who”…the closer we get to November 3, the more you should pay attention to the polls of Likely Voters (LV). Concurrently, the closer we get to election day, you should pay less attention to the polls of Adults (A) or Registered Voters (RV). Yes, it’s interesting to know what all adults think, and it’s good to know what registered voters think. But it’s GREAT to know what likely voters think because they’re going to ultimately decide the election.
As for the “how”…the best polls use live phone interviews — both cell and land lines. Including land lines is critical, especially in states with older populations. Otherwise the sample tends to skew young and not provide a truly representative glimpse of the electorate. That’s the same reason polls conducted entirely online are usually horrible — they just don’t capture enough older voters to provide accurate numbers. (For example, the three “D” pollsters I mentioned above received their low grades due mostly to their online format.)
Question 3: What is the poll’s margin of error?
Margin of error is probably the most important — and most misunderstood — aspect of political polling. Reputable polls will ALWAYS include a margin of error in the form of a plus/minus: +/- 3, for example. Rather than explain the statistical significance of the margin of error, let me demonstrate it.
Let’s say a poll shows former Vice President Joe Biden leading President Trump 51 to 43%. (FYI…this actually is about the national average as of October 4.) If the poll has a +/- 3 margin of error, that means the actual numbers could be 54 to 40% — a potential blowout. Or they could be 48 to 46% — a potential nail-biter.
As you can see, this is a huge swing. That’s why margin of error is so important to consider. It’s also one of the main reasons the 2016 Presidential Election results were so shocking. Too many Hillary Clinton supporters (and even her campaign insiders) were ignoring just how close the polling numbers were in the key electoral states of Michigan, Wisconsin, Pennsylvania, and Florida. Yes, she was ahead in most of the polls in those key states. However, her leads were often within the margins of error, so Donald Trump’s victories in those states was not as shocking as they seemed.
Two final points about margin of error. One, the best polls have a margin of error no higher than +/- 3 or so. Once you start getting a +/- of around 5 or higher, tread cautiously because that’s going to create some potentially huge swings in the actual vote counts. And, two, a “poll of polls” is an average and does not have a margin of error — even though each individual poll in it does.
Question 4: Is it better to compare different polls to each other — or to see how certain polls change (or not) over time?
This one is easy — it’s the latter. Because overall polling methodologies, samples, and wording of questions can vary widely even among the best pollsters, it’s really difficult to compare one poll’s results to those of another. Your best bet is to constantly compare current numbers from a poll to past numbers from that same poll. Only then can you truly tell if the trajectory of an election is changing. (Hint: this year’s Presidential election has been one of the most static in modern campaigns. Joe Biden’s national lead over Donald Trump basically has been the same for the past three months. That’s practically unheard of in today’s 24/7/365 news cycle era.)
Question 5: What about a poll with numbers that are very different from all the rest?
This probably is an example of a statistical outlier. As I mentioned in the introduction above, an outlier is an apparently random result that is not an accurate reflection of the current state of the race. As I also mentioned, even well-designed polls by historically reputable pollsters sometimes generate outlying results for no discernible reasons. Therefore, it’s best to focus on the average of polls. (Here’s the link to FiveThirtyEight’s national polling average for the 2020 Presidential race, which is updated multiple times a day.)
Having said that, an outlier sometimes is the first indication that a race’s dynamic is shifting. Therefore, you shouldn’t completely ignore one — just wait and see what subsequent polls have to say.
I hope this column helps you become a better, more informed consumer of political polls. Please share this column if any of your friends, family members, or co-workers would benefit from this brief introduction to Political Poll Reading 101.
For those of you wondering why I’m qualified to write about this topic, I have a Ph.D. in Human Communication Studies from the University of Denver — with an emphasis on leadership, power, and persuasion. My dissertation research focused on political communication, specifically Ross Perot’s 1992 Presidential campaign (“A Mythic Analysis of Ross Perot’s 1992 Campaign Infomercials as a Modern American Jeremiad”). And I have worked on — or consulted for — a number of political campaigns, ranging from the mayoral to the Presidential level.