polls being wrong doesn’t mean mathematics is bunk

Suggested listening: “Losing My Edge” – LCD Soundsystem


Perhaps this was the problem?

The writer Sam Kriss, in a March 2017 article for The Baffler, expressed his disdain for the polling industry, taking aim especially at coverage presented on the analyst Nate Silver’s website, FiveThirtyEight.com. Kriss argues that the polling-industrial complex, which includes media outlets, is doomed to repeat the sins of the past. The industry’s reaction to the existential crisis of polling—brought on, of course, by the unforeseen election of Donald Trump—has been insufficient, Kriss charges.

According to Kriss’ article, which bears the breezy title “Psephology in Free Fall,” Silver and the pundit class he represents have wrongly chosen to reexamine their research methodology rather than question their main operating assumption—namely, that we can obtain an accurate prediction of election results through opinion surveys with large sample sizes.

Kriss offers a Philosophy 101 critique of this idea, citing the variegated history of divination practices—for some reason the article revels in talk about birds being eviscerated—to suggest that Silver and his ilk are just the modern-day equivalents of oracles and shamans, with no better predictive ability. “When every poll gets it wrong, with increasing and alarming frequency, the problem is no longer methodological but metaphysical,” Kriss writes.

Although the basic thrust of Kriss’ attack is certainly correct—Silver was, in fact, systematically wrong about the 2016 presidential election—his advancement of this argument after Donald Trump’s victory suggests that he doesn’t believe his own conclusion—namely, that polling by random digit dialing has so little predictive value it might as well be tarot-card reading.

Kriss’ presumable point—that a preoccupation with favorable polls tends to engender political complacency—would have been a useful criticism before November 8, 2016. From the perspective of hindsight, it rings hollow. Why, in other words, didn’t Kriss think to say this earlier? If he—a high-profile writer with more than 20,000 Twitter followers—knew, on purely logical grounds, that the polls were wrong, why not say so while there was still time to do something about it?

As though anticipating this objection, Kriss points to the hostile reaction toward “qualitative” analysts, such as American University’s Allan Lichtman, who predicted Donald Trump’s victory, as a way of suggesting that speaking up would have been pointless.

On this score, I agree. Another “Trump’s going to win” hot take wouldn’t have mattered. It couldn’t have diffused the fog of moral certainty that surrounded the Clinton campaign.

But Kriss can’t embrace this defeatist logic, since it implies that his current anti-probabilist argument—another online take floating in the void of cyberspace—is also pointless. It seems only fair to assume that Kriss wants his contentions to be taken seriously.

The heart of the problem is that Kriss fails to consider the idea that, despite being imperfect, such dubious-sounding methods as augury, tarot-card reading, and, yes, even political polling still possess some predictive capacity, and, in fact, some of these techniques generate more accurate predictions—measured as average correctness over time—than others. In a shocking turn of events, it turns out that thinking about the future—even in a ritualized or religious mode—can, in fact, tell us things about the future. The accumulated wisdom of a sage is assuredly better than a broken clock. So, what’s really needed is a substantive critique of the specifics of Nate Silver’s approach to divination, and Kriss provides none.

Kriss invokes David Hume’s argument against causality—which states that the constant conjunction of events does not imply a causal relationship between them—to suggest that the exact future—the Event, to use the Lacanian terminology—is forever illegible, since its leading edge is always, by definition, just beyond the present.

One wants to ask if Kriss knows how to read a calendar. While it’s true that polling provides only a snapshot view of the stated opinions of a decidedly non-random group of people, that’s still some guide to what the future will look like. If 55% of 10,000 phone-answerers called on October 11, 2016, say they will vote for Hillary Clinton, then the pollster knows that, among phone-answerers in some geographic region, there are at least 5,500 people willing to say that they will vote for Hillary Clinton. That is not, by itself, enough information to predict the result of the presidential election, scheduled to take place 28 days later. Nevertheless, at bare minimum, we can infer that those 5,500 people are, on average, more likely to vote for Hillary Clinton than not. Using these methods, it may not be possible to assign a precise percentage to the chance that a given person will, in fact, pull the blue lever, but it’s pure mystification to assert, as Kriss dares, that polling provides no information at all.

In 2016, most of the signs pointed to a Hillary Clinton victory; it will be a “coronation,” we used to complain. And yet: Compare Kriss’ befuddling article to my November 6, 2016, blog post arguing that readers should vote for Hillary Clinton, the main objective of which was to discourage Leftists from voting for Jill Stein (or voting for Harambe, or not voting, or writing in Bernie Sanders, or whatever). My argument was very simple: Donald Trump is probably going to win because people—especially independent voters, who comprise the majority of the U.S. electorate—are justifiably unexcited by Hillary Clinton, which will cause low turnout for her side in key states. This is a concrete claim that offers clear reasoning as to why things will turn out a certain way. If the Event turns out the other way, then the reasoning was mistaken. There is no middle ground.

The real issue, in other words, is that Nate Silver-style prediction-making tries to answer every question that involves voting by taking a headcount. If there are X voters out there, and votes are apportioned across the determining units (states, counties, districts) like so, and Y people in State Z will vote for Hillary Clinton, then State Z’s electors (probably) will go to Hillary Clinton—and so on. Although this formulation of the issue is abstractly correct, the level of knowledge (and information-gathering) necessary to make it functional cannot be achieved: To get beyond a “good enough” prediction—a 99% confidence interval, say—one would have to be all but omniscient. Silver reasoned, plausibly, that large sample sizes and error-correcting transformations could overcome just about any problem with his model. That confidence is what turned out to be misguided, not the mathematics of probability as a whole.

There are, to be sure, a variety of ways in which polling can and does go awry—Trump supporters deliberately manipulating results, for instance! But that does not mean that gathering information is equivalent to staring into a crystal ball, even if statistical forecasting and palm-reading are, arguably, part of the same cultural tradition.

As a critique of punditry, then, Kriss’ article is a disappointment. As a philosophy, however, it is a dodgy invitation to incoherence. I won’t claim I saw it coming.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s