How Musings about Nate Silver are Misguided (and What That Means for Marketers)

By on in with 6 Comments

I’m a Nate Silver fan. Make no mistake. His book is at the top of my holiday reading list. I have been an avid follower of both his predictive models and his musings for the past 4.5 years. And, when my sister picked me up at the Austin airport on Tuesday evening and we headed to an election watch party, we wound up in a rather heated discussion as to the degree to which the results of the Presidential election were in doubt (I claimed that, as of Tuesday at 7:30 PM Central…they weren’t).

By late Tuesday night, analysts everywhere — despite their political affiliation — rejoiced. Tuesday night was a clear victory for math. The visualization below by the folk at Simply Statistics isn’t the prettiest thing in the world, but it shows how Silver’s predictions compared to the actual vote %:

That’s all well and good. As I said, a victory for math. The Simply Statistics post made an accurate statement when they wrote:

While the pundits were claiming the race was a “dead heat”, the day before the election Nate gave Obama a 90% chance of winning. Several pundits attacked Nate (some attacks were personal) for his predictions and demonstrated their ignorance of Statistics.

This is true. Mainstream media of all political leanings were motivated to drive viewership — explaining the first statement. Joe Scarborough will go down as the pre-election poster child for the latter statement. And, the night of the election, Karl Rove delivered the most memorable video on that front.

But, somehow, this “Silver vs. pundits” thing has taken an interpretive turn off course, as this is now being hailed as being the death knell for the punditry profession:

Hold the phone there, Leroy!

From Google, a pundit is:

An expert in a particular subject or field who is frequently called on to give opinions about it to the public

Pundits do a lot more than simply predict election outcomes. Political pundits, certainly, weigh in on how elections might turn out. But, they also weigh in on interpreting the electorate’s behavior, explaining and debating economic/foreign/healthcare policy, and proposing and defending various political strategies and tactics.

Nate Silver’s intent is to predict the outcome of the election based on the data available at the point of the prediction. One of Silver’s main, front page visualizations actually illustrates how his predictions changed over time:

The closer to the election, the more the probabilities drifted towards 100%/0%. This makes sense (and gets us to the main point of this post). There are two — and only two — underlying factors in the accuracy of the prediction at any point in time:

  1. The mindset/preference of all voters at that point of time — this is measured, quite accurately, by the slew of polls conducted throughout the election cycle
  2. The amount of time between that point and time and the election — the opportunity for “something to happen” that changes the mindset/preference of voters

Either one of these factors can drive the confidence (dangerous word to use here — that’s not in strict statistical significance meaning of the word) of the prediction up.

Pundits — who, often, also double as partisan strategists — actually weigh in on the interpretation of both of these factors:

  • The polls represent facts…but the facts have to be interpreted. And, that interpretation is with an eye to action. Which is…
  • What can “we” (strategist hat) or “they” (pundit hat) do to cause “something to happen” between now and the election to change voter mindset?

Michael Gerson weighed in on this with an op-ed piece that, while I don’t agree with his main conclusion, I think makes this point very eloquently:

The most interesting and important thing about politics is not the measurement of opinion but the formation of opinion. Public opinion is the product — the outcome — of politics; it is not the substance of politics.

Now, let’s pivot from political predictive analytics to marketers and marketing analytics. If a CMO goes to the CEO on Day 1 of a new quarter and says, “Our data is showing that we’re losing market share, but we’ve been working on a major new campaign that is launching next week, and we think we will turn the corner by the end of the quarter,” then he is making a reasonable claim. If, however, that same statement is made on Day 89 of the quarter…it is clearly poppycock.

Analytics — and predictive analytics in particular — are based on historical and “now” data. Analysts can and should be presenting “the truth” as to how things stand. They should be identifying specific problem spots so that, in collaboration with marketers, they can look to do something that will “make something happen” that drives more favorable results.

“Analytics vs. Punditry” is a false debate, as is “Analytics vs. Strategy.”

6 Comments


  1. Still mulling over this one. I think I’m in agreement with you. I guess my gut reaction is that I see Silver more as a journalist than pundit and journalists seek the whole truth as opposed to creating an argument and using facts to back that up. Silver starts with the research and reaches conclusions. Pundits work the other way around. Journalists seek the holy grail of (probably unobtainable) absolute truth. Maybe I’m off when I equate analytics with journalism but the statement that “analytics vs. punditry is a false debate” rings true. Pundits can stumble onto absolute truth but that isn’t their main goal.

  2. Maybe another way to look at it is that *one* role of pundits is to “prognosticate,” and Silver and his ilk have changed the game of prognostication to be much more rigorously data-driven. That makes pundits-who-prognosticate-based-on-their-hopes-in-the-hopes-of-changing-the-mind-of-the-electorate…need to stop doing that. They look silly and undermine their own credibility.

    While Silver now works for the NY Times, he’ll always be a sabermetrician to me. Just a sabermetrician who got sucked into political data.

  3. i am a Nate Silver fan way back before the NYT even knew who this young man was. That said i disagree that this is NOT the DEATH OF the PUNDIT. That is the PUNDIT as we know him/her to be today. Adapt or DIE is really the message of the election results.

  4. That’s a fair point. I think it’s fair to say that experts-with-agendas will continue to make a paid living and will continue to show up on TV and in print (digital or otherwise). The successful ones will need to become more data-savvy so as to not look utterly silly, but they’ll still exist.

Leave your Comment


« »