Let’s talk about the polls

📊 How pollsters are trying to come back from recent misses

This is right around the time in every election cycle that all kinds of data wizards and universities — including many you’ve probably never heard of — begin churning out polls, leading to a mess of numbers that probably does more to confuse than clarify about the direction of the campaign.

That’s happening to some extent this year, but there have actually been fewer polls than ever and a larger share from partisan sources — making it that much harder to separate out the quality surveys and get a sense of what’s going on.

So I wanted to call your attention to a few pieces written recently about the state of polling in 2022, especially since I’ve heard from many of you about your (understandable) skepticism towards polls after the industry’s recent misses.

The problems

It’s not just your imagination: It really is becoming harder to poll American elections. In a piece on Monday, “Frustrated With Polling? Pollsters Are, Too,” the New York Times consulted an array of pollsters to find out why. Here are some of the reasons they came up with:

• It’s becoming harder to predict who turns out to vote and in what numbers, especially after record-breaking turnout in 2018 and 2020.
• With fewer people using landlines and more people using caller ID, getting people to answer the phone for polls is harder than ever. (The team conducting a recent Times poll yielded about one completed interview for every two hours of dialing.)
• And then there’s the “Trump effect”: the disproportionate numbers of Trump supporters who just aren’t picking up the phone for polls, leading to what pollsters call “nonresponse bias.”

It’s that last problem that most bedevils pollsters. “The people you do get on the phone, you can always weight them up,” Republican pollster Jon McHenry explained to the Times. (Pollsters “weight” their surveys to give people in certain demographic groups more emphasis, in order to bring their pool of respondents in line with the electorate.)

But you can’t add more weight to people you haven’t even spoken to, meaning there’s no real way to correct for nonresponse bias.

The solutions

A piece in the Wall Street Journal on Monday, “Pollsters Try to Learn From Mistakes of 2016 and 2020 Elections,” discusses some of the novel ways pollsters are trying to address these issues.

Some pollsters are expanding the ways they reach voters. It’s already become common for one poll to reach voters by landline and by cell phone. Lee Miringoff, director of the Marist Poll, told the Journal that his outfit is also experimenting with using text messages to invite voters to take their surveys online.

Here’s another one I found interesting: “Celinda Lake, a Democratic pollster, said she has experimented with ways to build trust with voters so that they are comfortable talking with interviewers. In one political poll in Montana, her survey asked respondents if they had seen the Bobcats-Grizzlies game—a reference to the Montana State and University of Montana football teams.”

Lake explained that her goal was to drive up participation in the poll by including “questions that cue that you’re local, and not liberal.”

A different take

After 2016 and 2020, you might be approaching polling from the standpoint that it systematically overcounts Democratic support. That’s an understandable view to have.

In a piece last month, “Will The Polls Overestimate Democrats Again?,” FiveThirtyEight’s Nate Silver delved into the problem and found that polling bias has actually gone in both directions in the past decade. Yes, polls overestimated Democrats in 2014, 2016, and 2020. But they overestimated Republicans in 2012 and 2018.

Which isn’t to say that polls won’t overestimate Democrats again this year (very possibly for the reasons outlined above). But it’s useful to note that history shows a more complicated picture.

Of course, one hypothesis that comes up in all the pieces I’ve linked so far is that the 2016 and 2020 polling misses could have been a result of Trump being on the ballot. In 2018, without Trump, polls did really well and actually slightly overcounted Republicans; they were also accurate in the 2021 Georgia Senate runoffs.

Since there are only major elections every two years, we’re necessarily working off of a small sample size. We haven’t had many opportunities to see how polls perform in non-Trump years; that sets up 2022 as an important test for pollsters and their methodologies.

Reading the polls

Polls are also about how you interpret them. The Washington Post ran a piece this morning that offers help in understanding polls: “There are a ton of midterm election polls. Here’s a guide for how to read them.”

The Post recommends that you:

• Keep margin of error in mind and don’t pay too much attention to slim leads or small changes in the polls.
• Look at measures of voter enthusiasm, not just the overall number of who says they’ll vote for who.
• Be wary of outliers and partisan polls. If a number doesn’t sound right, it probably isn’t.

Let me add a tip: Pay attention to whether a poll is counting “likely voters” or “registered voters.” At this stage in the campaign, it might be more helpful to consider polls that are only looking at “likely voters,” rather than those with the wider “registered voter” pool.

Polls with a “likely voter” screen use statistical modeling, taking into account what a voter says about their likelihood to vote as well as data on their past voting behavior, to only factor in voters they judge as likely to actually cast a ballot.

Per FiveThirtyEight, if you were to only look at “likely voter” polls, the generic ballot average would show Republicans ahead by 1.1%, not 0.5%.  

More resources

If you’re interested in polling, in addition to the links above, I recommend the book “Strength in Numbers: How Polls Work and Why We Need Them” by G. Elliott Morris.

I don’t agree with all the takeaways in the books (at times, it’s a little poll-positive), but it provides a helpful history of polling in U.S. elections and an overview of where the industry is headed.

Plus, back in my podcasting days, I recorded a “user’s guide to polls” with MSNBC’s Steve Kornacki and FiveThirtyEight’s Nathaniel Rakich, which will help walk you through election surveys and how to understand them.

🗳 More midterm updates

Race spotlight: Pennsylvania Senate candidates John Fetterman (D) and Mehmet Oz (R) will meet tonight at 8 pm ET for their sole debate. Watch here

The Pennsylvania race is one of the most interesting on the map this cycle: it’s taking place in a key political battleground, it’s the most expensive Senate race of the year, and it’s one of the few competitive Senate contests without an incumbent.

It also includes two very compelling characters: Fetterman, a 6-foot-8, hoodie-wearing, tattooed lieutenant governor, and Oz, a celebrity doctor and daytime talk show host turned first-time candidate.

Neither is without controversy, from Fetterman’s 2013 armed chase of a Black jogger and his years of assistance from his parents to Oz’s carpetbagging and allegations of animal abuse.

After having a stroke in May, Fetterman’s health has also been a key issue in the race; it will be in the spotlight today, as he’ll be using closed captioning on the debate stage to see a live transcription of what Oz and the moderates are saying.

What I’m reading: “Pro-Trump Republicans court election volunteers to ‘challenge any vote’” (Washington Post)

• “Democrats growing anxious — again — over Black turnout” (Politico)
• “U.S. Allies Fear Trump Whiplash, Eye Midterms for Political Clues” (Bloomberg)

Bookmark: This NBC News tracker of the number of early votes cast and where they’re coming from. So far, more than 8.1 million votes have been cast.

Viral clip: Here’s a clip to watch from last night’s Florida gubernatorial debate:

🚨 What else you need to know

Ukraine: “A group of 30 House liberals is urging President Biden to dramatically shift his strategy on the Ukraine war and pursue direct negotiations with Russia, the first time prominent members of his own party have pushed him to change his approach to Ukraine.” (Washington Post)

• After an outcry, the House Democrats quickly put out a statement partially walking back their stance and clarifying their support for Ukraine.

Supreme Court: Justice Clarence Thomas temporarily paused a federal appeals court order requiring Sen. Lindsey Graham (R-SC) to testify before a Georgia grand jury investigating efforts to overturn the state’s 2020 vote.

• The order came from Thomas because he is the justice with jurisdiction over the appeals court whose ruling is in question. His “administrative stay” merely pauses things until the full Supreme Court has time to consider Graham’s request to overturn the lower court ruling.

United Kingdom: After being asked by King Charles III to form a new government, Rishi Sunak formally became the UK’s new prime minister this morning. Sunak, the UK’s first non-white leader, is the country’s third PM in seven months. At age 42, he is also the youngest British leader in more than 200 years.

Saudi Arabia: After surviving 15 presidents and seven kings, the U.S.-Saudi relationship is fracturing. Per the Wall Street Journal, Saudi crown prince Mohammed bin Salman has taken to mocking President Biden in private and “questioning his mental acuity.”

Recommended read:

Their America Is Vanishing. Like Trump, They Insist They Were Cheated.

The white majority is fading, the economy is changing and there’s a pervasive sense of loss in districts where Republicans fought the outcome of the 2020 election.

NYTimesBy

🗓 What your leaders are doing today

All times Eastern. Click on an event’s time to watch or listen to it.

Executive Branch

President Biden will receive his daily intelligence briefing (10:15 am) and then receive his second Covid-19 booster shot and deliver remarks on continuing to combat the pandemic (2:05 pm).

Vice President Harris will be in Albuquerque, New Mexico. She’ll deliver remarks at a fundraiser for Gov. Michelle Lujan Grisham (2:45 pm) and join Grisham for an event on abortion at the University of New Mexico (4:30 pm).

Second Gentleman Emhoff will be in Minnesota. He’ll participate in “Get Out the Vote” events in Eagan (1:30 pm), Edina (4:30 pm), and Minneapolis (6:45 pm).

White House press secretary Karine Jean-Pierre will hold her daily press briefing (2:45 pm).

Legislative Branch

The House will convene for a brief pro forma session (11 am). Such sessions are held only to fulfill the chamber’s constitutional obligation of meeting every three days; few members attend and no legislative business is conducted.

The Senate is not in session. Neither chamber of Congress is set to hold votes until November 14, after the midterms.

House Speaker Nancy Pelosi will meet with Israeli president Isaac Herzog (6 pm) at the Capitol, ahead of his meeting tomorrow with Biden.

Judicial Branch

The Supreme Court is not in session.

Campaign Trail

Here are the key races with debates tonight:

• Michigan Governor (7 pm): Gov. Gretchen Whitmer (D) vs. conservative commentator Tudor Dixon (R). [Polling average: Whitmer +5.4]
• New York Governor (7 pm): Gov. Kathy Hochul (D) vs. Rep. Lee Zeldin (R). [Polling average: Hochul +7.4]
• Pennsylvania Senate (8 pm): Lt. Gov. John Fetterman (D) vs. celebrity doctor Mehmet Oz (R). [Polling average: Fetterman +2.6.]
• Colorado Senate (8 pm): Sen. Michael Bennet (D) vs. construction company owner Joe O’Dea (R). [Polling average: Bennet +8.8]

👋 Before I go...

Here’s something sweet: President and First Lady Biden planted a ceremonial elm tree on the White House lawn on Monday to honor Dale Haney, the White House groundskeeper. The First Couple surprised Haney with the planting.

Haney has worked at the White House for 50 years and 10 presidents, dating back to the Nixon administration in 1972. He began as a gardener; he now oversees all 18 acres of the White House grounds.

He’s also known as the chief caretaker for the presidential pets, walking and looking after pets from Nixon’s Irish setter King Timahoe to Biden’s German Shepherd, Commander.

Here are more stories from Haney’s five decades at the White House.