Pollster reminds us that political polls are not infallible
In response to my opinion column about whether political polls have outlived their usefulness because of being off the mark in the 2016 and 2020 presidential elections, the director of the Floyd Institute for Public Policy at the Center for Opinion Research at Franklin & Marshall College in Lancaster wanted to try to set me straight on some of my conclusions.
Berwood A. Yost has written about measuring the accuracy of election polls and concludes that it is more complicated than determining whether the “polls got it right.”
If Yost is right, then you and I have misguided notions as to the purpose of these polls. We look at them as predictions of who is likely going to win a specific election. When the data do not match the results of the election, we become skeptical of the validity of the polls.
Yost suggests that there are numerous complications, namely: What’s the right way of measuring accuracy? How close to the final margin does a poll need to be to be considered accurate; is being within the poll’s margin of error close enough? What’s the life span of a polling estimate; how close to Election Day should a poll be conducted for it to count as a prediction?”
While these questions are valid, those conducting the polls fail to understand that the public is primarily interested in the final results, but they have limited interest in the supporting methodology and even less in explanations as to why the polls were off the mark.
The fact that news organizations cut to the chase when giving the results of these polls underscores how they want to engage the public. Certainly such important items as the margin of error, how many were polled, the dates of the polling and other critical components that go into the results are all in their own way important, but they are largely brushed aside by the public’s desire to know the bottom-line numbers.
Typically, it is these results that entice the reader to read the story. Headlines typically say something like “Biden leads Trump by 5% going into final week of campaign.”
Because of space limitations and other practical reasons, the headline does not include all of the caveats for the reader to take into account that make the headline not totally complete.
For one thing, the margin of error in many of these statewide polls is significant, from 4%-6%, so if the poll shows Biden leading by 5%, he might be trailing by 1% once the margin of error is factored in.
“If there is anything to take away from polling in recent elections,” Yost said, “it is that everyone should be more careful about making predictions based on a single indicator of who is ahead, particularly when there is so much other data we can use to tell the story.”
Yost believes that the primary issue that poll-takers confront is this: “What is the best way to talk about the results of a poll so that expectations about what the polling means are reasonable?”
Yost also believes that those conducting the polls must do a better job of reporting on the uncertainty of their estimates and discussing “ other potential sources of error beyond sampling, including the tendency for polls to share similar biases in any given election cycle.”
Yost says that it is important in pre-election polling to emphasize uncertainty. Last year, the average margin of error among the state-level presidential polls in 2020 was 3.9 points, he said.
Yost goes on to say that trying to assess the polls’ performance by looking at a single indicator is like judging the quality of a car by its paint color - easy to judge but meaningless until you understand the rest of the vehicle’s components and how the owner plans to use it.
“It is a bit baffling that so many people focus on a single indicator to assess accuracy, in this instance the horse race question that measures candidate preference,” Yost said. He adds: “Everyone needs to remember that polling is a helpful tool for reducing uncertainty, not eliminating it.”
This, however, flies in the face of what I said earlier: The public wants to know the results of the poll: Who’s ahead? Who’s likely going to win the presidential election?
We don’t want to have to plod through all of the fine print and qualifiers, so how about an accurate poll that answers the main burning question we have?
By Bruce Frassinelli | tneditor@tnonline.com
The foregoing opinions do not necessarily reflect the views of the Editorial Board or Times News LLC.