How Not to Be Fooled by Odds

Credit…Tomi Um By David Leonhardt

• Oct. 15, 2014

o 43

Now that The Upshot puts odds of a Republican takeover of the Senate at 74 percent, we realize that many people will assume we’re predicting a Republican victory. We’re not. There really is a difference between saying something will almost certainly happen and saying that it is more likely to happen than not.

We know many people don’t buy that, so let me explain. Maybe more important than any explanation, we have also come up with a list of outcomes that happen about 26 percent of the time — the flip side of 74 percent — which gives a sense for how common 26 percent still is. The list is below.

It’s an unavoidable truth that the world is an uncertain place. Probabilities help us make sense of that uncertainty. When a situation involves knowable odds and repeated trials, human beings often feel comfortable with probabilities. If I tell you that the odds of rolling an eight or lower with two dice are about 74 percent — 72.2 percent to be precise — you get it. You don’t think that I’m predicting you will roll an eight or lower. You won’t call me wrong if you roll an 11. You understand that I am saying you will roll nine or higher less than 50 percent of the time but more than never. The situation becomes more complicated when the odds are less knowable and the event in question is not an easily repeatable one, as is the case with an election. An election brings all kinds of uncertainties. Is the polling in sparsely populated Alaska accurate?

Will Kansans sour on an independent candidate after an initial flirtation with him? Will a late-breaking development hurt a candidate?

Not only are campaigns uncertain but we also can’t rerun them, as we can roll a pair of dice multiple times and see with our own eyes that 26 percent doesn’t equal 0 and doesn’t equal 50 percent. When an election is over, there is only one outcome, and it can come to seem inevitable — or at least come to seem as if everyone should have known the outcome all along. If only a few things had broken differently in 1960, for example, Richard Nixon might have won and John F. Kennedy might be remembered as an inexperienced loser.

Our forecast is as much a product of history as it is of math. We’ve looked at decades’ worth of election results and compared them with the current polls and other data. The combination indicates that the Republicans are in a strong position but not an invulnerable one. When candidates have a two- or three-point lead at this stage, as some do, history suggests they usually win. Put all that information together, and the Republicans have roughly a three-in-four chance of winning enough races to take the Senate.

Though we might wish otherwise, we can’t make uncertainty disappear. Traditionally, a lot of analysis and coverage of elections has made a version of this mistake. It’s been too quick to call a race a “tossup” even when one candidate had a clear, if still modest, advantage. To say that one candidate was ahead felt too much like making a prediction that could later look bad. Calling most campaigns a tossup or a close call was easier and safer.

But when you do take that approach, you’re not adding much useful information to the discussion. The only way to give useful probabilities — the kind that scientists, business executives and successful investors like Warren Buffett use in their worlds — is to give probabilities that will sometimes seem wrong. To be more specific, a prediction that puts a 74 percent chance on an outcome should be “wrong” about 26 percent of the time.

Here’s that list of situations that occur about 26 percent of the time (specifically, between 25 and 30 percent of the time). Not all of these are true probabilities; some are just frequencies. But they all give a sense for what 26 percent means.

■ The odds of rolling a 9, 10, 11 or 12 with two dice

■ The chances that a blackjack casino dealer busts

■ The percentage of calendar years since World War II that the Standard & Poor’s 500 has declined

■ The share of days in which it rains in Kansas City (as it did Monday, postponing a baseball playoff game)

■ The frequency with which a 25-year-old woman is shorter than 5 feet 3 inches

■ The frequency with which a 25-year-old man is 5-11 or taller

■ The odds that a National Football League defense prevents a first down on third-and-one.

■ The percentage of full-time graduate students in electrical engineering in this country who are American citizens

■ The percentage of mothers with children under 18 who stay home with their children

■ The share of Americans who live in California, Texas or New York

■ The percentage of subway cars on the C line with dirty seats or floors, according to the Straphangers Campaign

If the odds of a Republican victory — in our Senate model and others — remain in the neighborhood of 74 percent, it won’t be right if the Republicans win any more than it will be wrong if the Democrats keep control. These models should instead be judged over multiple elections and years: When they say 74 percent, have they made you smarter about the state of a campaign?

Amanda Cox and Josh Katz contributed reporting.