American pollsters have a big problem. An industry that needs credibility to make money is unable to accurately predict election results. What’s more, the pollsters aren’t off by just a little bit. The 2020 pre-election polls compared to the actual results were worse than the 2016 debacle.
The 2016 results weren’t as bad as 2020 as far as the margin of error between the final polls and the actual results. But 2020 was the worst performance by many of these respected national polls in decades.
A few examples: the polling average for Biden in Wisconsin before election day 2020 was a robust +8. When the votes were counted, Biden had won by less than a percent. In Florida, pre-election polls had Biden at +2. Trump won the state by +3. In Pennsylvania Biden was at +5 before the election but +1 after.
Polling may not be a “science,” but it is based on scientific principles — statistical probability, random sampling, and accepted scientific methodology. No good pollster puts his thumb on the scale and attempts to give a client an answer he thinks the client wants to hear. So why have the polls been so out of whack with reality the last few election cycles?
The top numbers man at the New York Times, Nate Cohn, says “alarm bells are going off again” because “Democrats are polling well in exactly the places where surveys missed most in 2020.”He says “Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.”
The simple reason, according to Robert Cahaly of Trafalgar Group, is that pollsters aren’t talking to the right people. Cahaly’s polling has been head and shoulders above almost all others, and National Review‘s Dan McLaughlin wanted to understand why.
As I and others at National Review have detailed previously, Trafalgar has racked up a string of polling successes in recent years by talking to the very sorts of voters that other pollsters have missed, many of whom have responded to Trump and Trumpish candidates.
Trafalgar was ahead of the curve in Virginia in 2021, and alone in seeing a tight race in New Jersey in 2021. In 2020, even when Trafalgar got the outcome wrong in projecting narrow Trump victories in some states, his forecasts were often closer to the result than competitors who projected comfortable wins for Biden in states that Biden carried by his fingernails. Trafalgar has also turned in strong performances in this year’s Republican primary polls.
Trafalgar’s most recent polls seem almost suicidally wrong. They have Kari Lake winning by four points in Arizona. Dr. Oz and Doug Mastriano are each down two points in Pennsylvania; Lee Zeldin is losing by five in the New York governor’s race. Tiffany Smiley is down three in the Washington Senate race, while Brian Kemp is up seven and Herschel Walker is up one in Georgia.
Other polls show those candidates losing by much larger margins. But Cahaly’s outfit has a unique way of reaching out to respondents, recognizing their reluctance to participate in polls. “Polling is broken,” he told McLaughlin. And perhaps his success in being more accurate than others stems from the fact that he actually seeks out voters whom most pollsters miss.
In particular, Cahaly stresses that he has to be more creative in approaching voters not only due to self-selection effects and the diminishing share of Americans who have landlines and answer the phone, but also due to what he calls “submerged” voters: people who are afraid to give their political opinions to strangers for fear of being doxxed or, in the post–January 6 environment, federally investigated. Needless to say, voters with these sorts of concerns are disproportionately not Biden supporters. “I’ve never seen anything like that,” Cahaly says of the number of responses he is getting now from people who need persuading that they are talking confidentially to a legitimate pollster. “This is real. People are scared.”
Regardless of methods, a pollster can only be judged by results. And since most pollsters have yet to solve the riddle of “What went wrong” in 2016 and 2020, perhaps Trafalgar’s approach could improve the end product. Since polling is inexact, that may be the best we can hope for.