How Polls Should Be Interpreted, or Not

1 General Advice

Polls do not tell us what’s going to happen on election day. Especially polls taken months in advance. Typical polls don’t even ask about election day; instead they ask what would happen “if the election were held tomorrow” which of course it won’t be. A lot can change between now and November.

Polling data may give us a general outline of what “could” happen. However, even if the outline is correct, the details probably won’t be. The more detailed the prediction, the less likely it is. See item 7 below.

Polling data may give us some idea of where things stand at the moment. Recent trends may shed some light on which tactics are working and which are not.

Do not over-react to the results of a single poll. The average over many polls (here, or here) is much more meaningful.

Anybody who looks at poll numbers and grows complacent (or, conversely, despondent) is a fool. The fact is, issues matter, candidates matter, and campaigns matter. The whole point of campaigning is to cause a shift in public opinion.

In particular, it is important to contest every race. Vigorously. Even if you don’t win this time around, you lay the groundwork for winning in some later cycle. You have to win the argument before you can win the election.

Some politicians seem to have given up on trying to shift public opinion; instead they obsess over turning out their base and/or suppressing the opposing base. I consider this political malpractice on a epic scale. It’s not a winning strategy in the long run. And even if it works in the short run, it damages the foundations of democracy.

There is such a thing as poll porn. Every so often we are allowed to take a break from the trench warfare and enjoy some poll porn. Don’t take it too seriously.

Polling data (along with historical data) can be fed into statistical models that try to predict the outcome of the election. This is a very, very complicated undertaking. It involves considering thousands or even millions of possible scenarios. Nate Silver knows how to do this. Very few other people do.

Figure 1: Modeling Possible Election Outcomes

In 2016, the raw polling data was correct. The problem is, a great many people misinterpreted the polls, including people who should have known better (e.g. the Hillary campaign) and people who had no business interpreting polls at all (e.g. James Comey). A few people got it right, including Nate Silver. I knew he was right, partly based on my own models, and (!) because he showed his work. I said so at the time, as loudly as I could.

There is a huge market for “internal” polling, which is notoriously rife with abuse. In contrast, most public pollsters have a brand and a reputation that they need to defend. Before reading any poll, check the pollster’s rating. Over time, bias and/or will be detected.

10.

Prediction markets have a fair bit of predictive value. Unlike polls, they ask questions that focus on the real election (rather than the counterfactual “if the election were held tomorrow”).

11.

The action item is clear: Run up the score. Run it up everywhere. Run it up so high that no “polling miss” will make a difference, and no amount of cheating will make a difference.

Flip the senate and the state legislatures also.

2 Sources of Information

There are tremendously many pollsters. To find out which are reputable and which are not, check the FiveThirtyEight Pollster Ratings.

It’s painful to track down the primary source for each poll, so you may find it helpful to rely on aggregators. Most of them allow you to filter out which race(s) you are interested in: presidental primary, general election popular vote, presidential state-by-state, senate, or whatever. Some good, widely-cited aggregators are:

In addition, we have pundits, who look at polling data and mix in their opinions. For example, in mid-2020, the Biden was ahead by several points in Florida, but most pundits rated it a toss-up, because the state party is so screwed up that they were likely to find a way to blow it.

A useful aggregation of aggregators and pundits, with a convenient way to visualise the results, can be found at 270ToWin. There is a pulldown that allows you to select which data-source you want to map out.

Then there are the prediction markets: