Why Were So Many Experts Surprised by Obama's Victory?--UPDATED

Anchorage--

A number of people have asked why so many presumably well-informed people--apparently including high-ranking people in the Mitt Romney campaign itself--were so surprised when President Barack Obama won re-election, particularly by the margin he did.    One of those asking was my sister, so I wrote this analysis for her.    I am taking my wife's suggestion to post a slightly edited version of that analysis on this blog.    

Despite a number of public polls showing the likelihood of an Obama victory, some analysts gave wildly incorrect predictions of a strong win for former Gov. Mitt Romney, the Republican nominee.   One example of such a terrifically inaccurate prognostication was that by the long-time political analyst Michael Barone.   This veteran columnist and co-founder of the seminal Almanac of American Politics wrote on Nov. 2 that Romney would win 315 electoral votes.   Barone gave a backup prediction in the same piece that subtracted Pennsylvania and Wisconsin from Romney's column and thus still had Romney winning the presidency with 285 electoral votes, 15 more than the 270 needed to gain victory in the Electoral College.   (Barone eats his crow here in the face of the President's victory, which now with the Florida recount showing Obama the winner in that state  makes his Electoral College total stand at 332.)  

This question of "Why were people surprised when Obama won?" is separate but related to the question of why Obama won, which would include factors such as the nation's changing demographics, the Obama campaign's superior get-out-the vote efforts (as conceded by Romney campaign staff), and the apparently more effective use of advertising by the Obama campaign and its affiliated groups.    Influenced by the commentators Josh Marshall and James Fallows, I offer this list of reasons for why there was so much surprise Tuesday night in the world of Fox News:

1.   As my son's Campaigns and Elections professor pointed out to that class, the economic fundamentals were not as negative for Obama as a number of commentators thought, particularly because the economy is improving and many voters blame George W. Bush for the disappointing economy.

2.   Some commentators predicting a Romney victory focused on a perceived momentum and an intensity gap favoring Romney in the last month of the campaign.   The problem was that at least in some cases this perception of momentum, enthusiasm, and intensity seemed to be sharply influenced by the facts that (a) those commentators themselves strongly disapproved of Obama's policies and (b) those commentators spent a lot of time talking with other people who strongly disapproved of Obama's policies and projected the feelings of their friends to the electorate as a whole.   (This last sentiment is sometimes known as "the living in a bubble and not knowing it" problem, (unfairly) associated with the former New Yorker film critic Pauline Kael's alleged reaction to Richard Nixon's sweeping victory in 1972.)

3.   More technically, some commentators thought that most of the polls showing a lead for Obama were skewed by assuming that the voting public would consist of more Democrats than Republicans.   Those commentators thought that was unlikely.  A good example of that "Most polls are skewed" view was the John Podhoretz column just before the election in the New York Post which relies on the Rasmussen and Gallup national polls, two organizations that consistently reported better results for Romney that many other polling outfits.   As Podhoretz notes, Rasmussen predicted in October 39 percent R and 33 percent D, and Gallup predicted in October 36 percent R and 35 percent D.

The exit polls showed that most of the polls conducted by organizations other than Rasmussen and Gallup were right--in fact, 38 percent of the voters identified as Democrats and 32 percent identified as Republicans.   

How did Rasmussen and Gallup get this so wrong?

I have heard four explanations.    The first is a huge racial mistake by Gallup, which assumed that 78 percent of the voters would be white.   Instead, the exit polls showed that only 72 percent of the voters were white.   Nonwhite voters were obviously much more likely to vote for Obama than white voters.

A second factor is that Rasmussen and Gallup missed how much self-identification as Republican has dropped in recent years, with possible explanations being unhappiness with George W. Bush's presidency or displeasure with perceived obstructionism by Congressional Republicans during Obama's presidency.

The other two explanations for why Rasmussen and Gallup thought the voters would be so much more Republican and/or white than actually occurred are rooted in technical deficiencies in the methodologies of some polls.   Rasmussen runs robo-polls through automated pre-recorded telephone calls, and federal law prohibits a computer from calling a cell phone.    People who have only cell phones and not land lines tend to be younger and more likely to be minorities than people who have only land lines.    It also appears that some polling organizations do not use Spanish-language interviewers, and that might produce a small but significant bias in critical states such as Florida and Colorado.

4.   Some commentators predicting a Romney victory pointed to the traditional rule in politics that undecided voters break late against the incumbent.   That traditional rule did not apply in 2012.   In fact, exit polls showed that most of the 9 percent of the voters deciding in the last few days went for Obama.       

5.   Romney led in many of the polls (and in the exit polls) among independents, and some commentators predicting a Romney triumph cited the traditional rule in politics that the candidate who wins independents wins the election.   Independents did go for Romney--the exit polls showed that Romney won independents by only 5 points, although that was a smaller margin than the Romney campaign was counting on.   But that 5-point advantage was also insufficient for Romney because of another problem with the traditional rule this year alluded to in Point 3 above--it is not true that independents are a group sitting equidistant between the two parties.   A number of self-identified independents are former Republicans who had stopped using that label.  As Josh Marshall has pointed out, a number of those newly self-identified independents were still likely to hold conservative views and may have stopped labeling themselves as Republicans out of a belief that the Republican Party favored too large a government.   That movement of Tea Party types away from describing themselves as Republicans weakened the predictive effect of the "independents control" rule.     

[November 11--Following a conversation with my son, this morning I updated this post and tweaked it for clarity.]