9 min read

How did the polls do?

The polls did well this election cycle. It was me and my fellow pundits who messed up.
How did the polls do?

Good morning! It’s Monday, November 14, 2022. The 2024 elections are 722 days away.

This morning, I have lots of reporting for you on the latest midterm results — and the post-election fallout in both parties.

But first: I often like to use the newsletter to answer questions I got from readers, this is one I get a lot: can we really trust the polls? So, now that the dust is beginning to settle after last Tuesday, I wanted to take this morning to look at how the 2022 polls — and punditry — fared.

The piece also offers a chance to do some introspection about my own coverage, which I like to do from time to time. In my view, that’s one of the benefits of an independent newsletter: instead of using the never-wrong, voice-of-God approach you might get from a more traditional newspaper, this format allows for more of an informal conversation between writer and reader, where I can be honest with you on the occasions that I might fall short.

📣
If this newsletter was forwarded to you, subscribe here. If you want to contribute to support my work, donate here.

Don’t blame the polls. It’s us pundits that were off this year.

The midterm elections are (mostly) over, which means we’ve arrived at America’s favorite biennial exercise: complaining about the polls.

After a “red wave” failed to materialize last week, I’ve seen a lot of tweets and posts from across the political spectrum complaining that voters were misled and misinformed by pre-election polls. Maybe, some of them have suggested, we’d be better off ditching the polling industry entirely.

In a Financial Times piece that was widely shared on social media this weekend, journalist Edward Luce argued that American media broadly whiffed on its election coverage once again — blaming the “increasing big media tendency to substitute opinion polls and the predictions of data aggregators for real political reporting.”

So, were the polls really that off? Let’s take a look. Here are the FiveThirtyEight polling averages for key Senate races on October 8 (one month before Election Day), November 8 (Election Day), and the current results as they stand this morning (with the caveat that there are still some more votes to be counted in each race):

And here is the same chart for competitive gubernatorial races:

One thing that stands out to me, and perhaps to you, is that, in many cases, the polling average on October 8 was closer to the final result than the average on November 8. As it turns out, the flood of partisan polls from Republican-leaning firms that came gushing out in the final days of the campaign did incorrectly bias the polling averages towards Republicans in the last month — as warned about in this newsletter and many other outlets.

That phenomenon shows how polling can be vulnerable, just like any other source of information. As I have also written previously, it has always been wise not to buy into individual polls completely, but rather to put them into an average to get a better sense of where a contest stands. But, all in all, if you look at how the polling averages ended up — I don’t think it’s anywhere near the crisis for polling that some commentators are making it out to be.  

For the most part, as you can see above, the polls landed fairly close to the final result — perhaps off by a few points in one direction or the other, but generally giving a good idea of which races were close and which party held an edge. (A polling edge of just one point, remember, should never really be regarded as an edge for one party but rather an indicator that the race is very close and could wind up either way.)

So where did the “red wave” narrative come from? Not the polls, but from me and my colleagues: the pundits. Strangely, Luce and other observers’ response to the supposed polling miss this election seems to be suggesting more punditry: a return to less data-driven journalism and more anecdotal reporting, like the “diner stories” that became a journalistic leitmotif of the Trump era.

But we would have been better off in this election — and will likely be better off in elections to come — if we trust the data more, not less. There are several cases this cycle where the pundit narrative got ahead of the data, promulgating the idea of a “red wave” as a result.

In the Arizona governor’s race, for example, various pundits hailed Republican nominee Kari Lake as “the next Republican star,” Trumpism’s new “leading lady,” and a possible future vice presidential candidate. Democrat Katie Hobbs, meanwhile, largely ducked the media attention that Lake seemed to delight in; her campaign was frequently portrayed as doomed. But, through this all, polls actually showed Lake and Hobbs neck-and-neck — and that is where the race has landed. (Hobbs has a small edge and the upper hand as of right now, but no outlets have declared a winner in the race.)

Or take the Pennsylvania Senate race, where pundits began to change their view of Democrat John Fetterman’s chances after a shaky debate performance last month, brought on by complications from his stroke in May. To use a word that has become all the rage in political journalism — but perhaps should be retired from the reportorial lexicon — the vibes turned harshly against Fetterman towards the end, but the polls did not follow. There was some movement towards Oz (partly driven by the same partisan polls that appeared in other races), but the race remained close to the end; Fetterman eventually flipped the seat.

In fact, daily favorability tracking suggests the debate may actually have helped Fetterman — an instance where the data seems to have picked up on a more accurate trend than the journalists. In both Arizona and Pennsylvania, the vibes were wrong, but the data was largely right.

The vibes said Lake. The polls said otherwise. (Gage Skidmore)

In our defense, the data in this election was just plain weird, as I tried to communicate throughout the cycle. There were two conflicting stories that kept showing up. In one of them, poll after poll showed Americans identifying the economy as their top issue — and disliking the state of the economy and trusting Republicans more to handle it. When combined with the historical trends that have persisted in almost every midterm, the “red wave” narrative became an easy one to write.

But the polls in individual races told a different story, repeatedly showing Democrats performing better than the “fundamentals” would suggest. Seeing as polls have been slightly biased towards Democrats in recent cycles, and that more nuanced bucket of numbers conflicted with the clear historical patterns, many in the press corps (again, myself included) chose to focus on the set of data that aligned with normal midterm expectations.

As it turns out, both stories were correct. According to exit polls, Americans did care about the economy — but, to an unprecedented degree, voters who disliked the economy and didn’t approve of the sitting president were willing to vote for the president’s party anyways, seemingly because of messages around abortion and democracy that many of us in the media dismissed. We were wrong. We should have just trusted what the numbers were telling us, contradictory as they may have seemed.

In one notable example in late October, a set of New York Times polls of individual House races showed promising results for Democrats — but were labeled in the Times as “fresh evidence that Republicans are poised to retake Congress,” because the polls also showed that Biden was unpopular and the economy was a top concern. The results of each contest ended up being pretty spot-on with what the individual race polls had said.

Polling is not perfect, and anyone consulting polls should be aware of their potential pitfalls. (This cycle especially shows to be skeptical of partisan polls and to trust the averages most of all.) But in this election cycle, journalists would have better served our readers if we had shed our faulty assumptions and hewed more closely to what the data was telling us, rather than injecting our own priors (and punditry) into it.

On-the-ground reporting, like the kind Luce recommends, is important too — but there is no evidence that random man-on-the-street (or man-in-the-diner) interviews lead to a better sense of the nation’s mood. The age of anecdotal-only journalism was not preferable to the scientific methods we have now, methods that performed fairly well this year.

I and many other journalists made mistakes this year when we ignored telling data that didn’t comport with our expectations or historical trends. Let’s not make the mistake worse by seeding unnecessary mistrust in polls and veering away from the data even more.


🚨 What else you should know

There are a lot of other fascinating dynamics coursing through American politics right now — and I’ll be covering them all in the days ahead, during what promises to be a consequential week. Here’s a quick look at some of the storylines that will be driving the week.

➞ The Democrats have won the Senate. Nevada Sen. Catherine Cortez Masto’s re-election (declared on Saturday) handed Democrats their 50th seat, guaranteeing two more years of Democratic control — and Biden-nominated judges. The Georgia runoff on December 6 will tell us whether Democrats will end up netting a seat, or if the Senate will stay at 50-50.

➞ We’re still waiting on the House. The AP has called 212 races for Republicans and 204 for Democrats, putting the GOP six districts away from claiming a House majority. Here’s a handy NYT tracker of the remaining 19 races, which are largely concentrated in slow-counting California. Several batches of votes that came in Sunday seemed to damper Democratic hopes that the House could still be in reach — but control of the chamber is poised to be close either way.

➞ A consequential lame duck awaits. As the election results get finalized, both chambers of Congress return to Washington this morning for the “lame duck” session, which takes place between Election Day and when the new crop of lawmakers are sworn in come January.

This year’s lame duck is likely to be packed with action, with big-ticket items like legalizing same-sex marriage, raising the debt ceiling, and reforming the Electoral Count Act on the table.

➞ Election deniers were denied. With the defeats of Jim Marchant in Nevada and Mark Finchem in Arizona, every single election-denying secretary of state candidate running in a political battleground lost last week. That outcome is a crushing blow to Trumpism, coming as...

➞  An existential week for the GOP looms. Former President Donald Trump is widely expected to announce a 2024 presidential bid tomorrow, even as Republican pols are seeking their distance from him and eyeing other options.

Meanwhile, the House GOP is scheduled to hold leadership elections tomorrow and the Senate GOP is scheduled to follow Wednesday — but there is growing pressure to reschedule as the party thinks through where it stands. An outpouring of Senate Republicans, including Sens. Ted Cruz (R-TX), Lindsey Graham (R-SC), Josh Hawley (R-MO), and Marco Rubio (R-FL) have endorsed delaying their elections, a sign of the deep anger within the party at once again losing the majority.

I’ll be covering all of these stories in more detail later this week, so stay tuned for more.


🗓 What your leaders are doing today

All times Eastern. Click on an event’s time to watch or listen to it.

Executive Branch

President Biden is in Bali, Indonesia for the G20 Summit, a meeting of the leaders of the world’s largest economies. Earlier this morning, he met with Indonesian president Joko Widodo and — in a highly anticipated first face-to-face sitdown since Biden took office — with Chinese president Xi Jinping.

He also took questions from reporters at a press conference.

Vice President Harris will ceremonially swear in Candace Bond as the U.S. ambassador to Trinidad and Tobago (2 pm).

First Lady Biden will travel to Illinois with Commerce Secretary Gina Raimondo, Labor Secretary Marty Walsh, and Education Secretary Miguel Cardona to kick off National Apprenticeship Week.

The four officials will visit Rolling Meadows to meet with students in the local high school’s Career Pathways program (1 pm) and hold an event in Chicago to highlight successful Registered Apprenticeships programs (2:45 pm).

Legislative Branch

The Senate will convene for the first time since the midterms, after a month of recess (3 pm). The chamber will hold one roll call: a cloture vote to advance the nomination of María del Rocio Antongiorgi-Jordán to be a U.S. district judge in Puerto Rico.

The House will also convene for the first time in a month (2 pm). The chamber is scheduled to vote on 29 bills, most of them renaming post offices or VA clinics.

Judicial Branch

The Supreme Court will release orders in various pending cases (9:30am).


👍 Thanks for reading.

If you enjoy Wake Up To Politics, it’s always appreciated if you donate to support the newsletter or buy some merch. Or if you tell your friends and family to sign up at wakeuptopolitics.com.

If you have any questions or feedback, feel free to email me: my inbox is always open.

Thanks for waking up to politics! Have a great day.