2012 Presidential Prediction Rankings

This chart ranks how well 25 major forecasters predicted the outcome of the 2012 presidential election. Obama won the election 332-206, with a 3.9% popular vote margin.

#	Predictor	Outcome	Obama / Romney picks	Pop=3.9%	StateFail	Grade
1	Markos Moulitsas and Daily Kos Elections	332-206	NV PA MN WI IA NH OH CO VA FL	3.5%	0	A+
2	Nate Silver, New York Times	332-206	NV PA MN WI IA NH OH CO VA FL	2.5%	0	A
3	Simon Jackman, Huffington Post	332-206	NV PA MN WI IA NH OH CO VA FL	1.7%	0	A
4~	Josh Putnam, Davidson College	332-206	NV PA MN WI IA NH OH CO VA FL		0	A
4~	Drew Linzer, Emory University	332-206	NV PA MN WI IA NH OH CO VA FL		0	A
6	Sam Wang, Princeton University	303-235	NV PA MN WI IA NH OH CO VA FL	2.34%	0.9	A-
7	Jamelle Bouie, American Prospect	303-235	NV PA MN WI IA NH OH CO VA FL	2.2%	0.9	A-
8~	TPM Polltracker	303-235	NV PA MN WI IA NH OH CO VA FL	0.7%	0.9	A-
8~	RealClearPolitics	303-235	NV PA MN WI IA NH OH CO VA FL	0.7%	0.9	A-
10	Intrade Prediction Market	303-235	NV PA MN WI IA NH OH CO VA FL		0.9	A-
11~	Ezra Klein, Washington Post	290-248	NV PA MN WI IA NH OH CO VA FL		3.9	B
11~	Larry Sabato, University of Virginia	290-248	NV PA MN WI IA NH OH CO VA FL		3.9	B
13	Cokie Roberts, ABS NEWS	294-234	NV PA MN WI IA NH OH CO VA FL		5.6	B
14	Dean Chambers, Unskewed Polls	275-263	NV PA MN WI IA NH OH CO VA FL	1.79%	10.5	C+
15	Erik Erickson, Redstate	285-253	NV PA MN WI IA NH OH CO VA FL		17.2	C
16	SE Cupp, MSNBC	270-268	NV PA MN WI IA NH OH CO VA FL		22.0	C-
17	Karl Rove, Bush advisor (popular)	285-253	NV PA MN WI IA NH OH CO VA FL	3%	23.9	D+
18	Ben Shapiro, National Review	311-227	NV PA MN WI IA NH OH CO VA FL		28.0	D
19	Ben Domenech, The Transom	278-260	NV PA MN WI IA NH OH CO VA FL	R	26.7+ME2	D
20	Christian Schneider, Milwaukee Journal-Sentinel	291-247	NV PA MN WI IA NH OH CO VA FL		30.5	D-
21	James Pethokoukis, AEI	301-227	NV PA MN WI IA NH OH CO VA FL	2%	30.5	D-
22	Michael Barone, Washington Examiner	315-223	NV PA MN WI IA NH OH CO VA FL		33.8	F
23	George Will, Washington Post	321-217	NV PA MN WI IA NH OH CO VA FL		35.7	F
24	Steve Forbes, Forbes Magazine	321-217	NV PA MN WI IA NH OH CO VA FL		40.5	F
25	Dick Morris, Fox News	325-213	NV PA MN WI IA NH OH CO VA FL		41.5	F

Here are the ranking methodology and criteria for inclusion. I'm only putting professional pundits and big things like Intrade on the chart. But I should additionally mention that my co-blogger Nicholas Beaudrot got every state right.

The rankings are based on how well people predicted individual states, with the penalty for missing a state being the percentage margin of victory. This penalizes people less for getting close states wrong, and more for losing big. The values of the various states, via Huffington Post, are MN=7.7 WI=6.7 NV=6.6 NH=5.8 IA=5.6 PA=5.2 CO=4.7 VA=3.0 OH=1.9 FL=0.9.

I use accuracy of popular vote prediction to break ties. I'm working with a popular vote margin of 3.9%. Since many people didn't try to predict the popular vote, this is somewhat artificial. So I've added the letter-grade component on the right, which doesn't take popular vote into account, except in the case of the Daily Kos folks who get an A+ for getting the popular vote closest to right.

For now, I'll outsource commentary on the success of Nate and the other poll aggregators to xkcd:

Prediction-Gathering Post

Happy Election Day, everyone! Now that things are up and running, I'm putting up a post to gather people's election predictions.

I'm intending this site to be more a ranking of the punditocracy than a fantasy football league for all of us, so what I'm asking for are links to professional pundits. (But if you'd like to have your election predictions recorded for posterity in this thread, feel free to put them there!) I see that Brad Plumer's prediction-compiling post has Nate Silver, Sam Wang, Drew Linzer, Michael Barone, Ezra Klein, Larry Sabato, Josh Putnam, Jay Cost, Philip Klein, Ross Douthat, Jamelle Bouie, George Will, Ben Domenech, Markos Moulitsas, Karl Rove, James Pethokoukis, Dick Morris, Jim Cramer, and Dean Chambers in it. There's also a National Review prediction thread. Where else should I look? I'll try to use the last predictions people made before the first ballots of November 6 were cast.

As I suggested in the methodology post, I need people who have come up with an electoral map and preferably a popular vote margin as well. They should also be reasonably famous political prognosticators who are cited in print or televised media. A Facebook friend suggested that I rank P'Lod the All-Seeing, an alien quoted in the Weekly World News. But unfortunately, P'Lod didn't specify an electoral map.

Welcome To VoteSeeing!

We're in the last days of the 2012 presidential election, and we're about to see a lot of predictions. I've started VoteSeeing to help us keep track of the predictions and see which ones were more accurate than others.

I've been talking with a few smart mathy friends about this, and my plan is to rank people's presidential election predictions by the two following criteria:

1. Summed State %Fail: I'll consider all the states where people didn't pick the winner, and add up the winner's percentage margin of victory in each of them. Lowest score is best here, with 0 meaning that you didn't miss a single state.

2. Popular %Fail: The percentage by which people missed the popular vote. When people get the same score on 1, probably because they picked an identical electoral map, I'll use this to distinguish between them. Again, lowest score is best, with 0 meaning you hit it exactly.

If someone doesn't pick a winner in a state, I'll just count half the margin of victory against them. That way, correct prediction is better than saying nothing, which in turn is better than error. If somebody doesn't say anything about the popular vote, I'll use a default value of the popular vote being tied (but don't worry, I'll mark this clearly, because it's totally artificial). 

Why this method? And in particular, why do 1 this way? Consider the following example: Donkey picks Obama to win Florida and Alaska, while Elephant picks Romney to win both states. And suppose Obama wins Florida by 0.1%, while Romney wins Alaska by 30%. I'd consider Elephant to be the better predictor here. Florida was really tight and could've gone either way. Elephant made a reasonable guess that was almost right. Donkey, meanwhile, was way off on Alaska in a way that just looks silly. Donkey, however, got closer to the total electoral vote, since Florida is worth a lot more.  So if you count based on that, you end up with the result that Donkey is the better predictor. And if you just count based on how many states people missed, you get them being equal. The method I've suggested rates Elephant higher in this case, which I think is the right answer.

In comments below, people have suggested methods that would give more weight to Florida than Alaska, since it's worth more votes. I think this is a very reasonable suggestion, and I've been chatting with friends about it. But in the end, I've stuck with the percentage system.  To see why, look at the results from 2008. Do we want to penalize someone more for saying Obama would win Alaska, or that McCain would win Florida? I think Obama in Alaska is the sillier pick. The %Fail system delivers this result, while summing vote counts says that Obama in Alaska is better, since you were off by fewer voters. Nevertheless, once raw vote counts stabilize, I may add in those numbers as well, as it's also a nice metric. And in the end, I doubt this will make a huge amount of difference.

There might be a lot of more sophisticated ways of doing this, if people were making more complex predictions. If people were picking an amount by which candidates would win each state, we could take the sum of the error (or maybe the sum of the square of the error). But usually all people give you is a map and a popular vote percentage, so I'll be working with that.

Maybe I'll do things with Senate predictions later on if people are interested. But for now, just the presidential election. This is going to be interesting!

Political blogging elsewhere

This is one of my many political blogs. Here are a few of the others, on the off chance that you're interested. I'm putting this low on the page, so people will run into it only after they see the stuff this blog is actually about.

My main blog these days shares a name with me. I mostly do my political commentary on Facebook these days, since that's the place to reach people and get comments. I export some of it to the blog if I think people might have some interest in seeing it. 

From 2004-2015, my homepage on the internet was called The Ethical Werewolf. (The first post there will explain the picturesque name). It started out with lots of posts about politics and philosophy, as well as some random stuff about my life. In later years it became a repository for wacky philosophy puns.

From 2008-2015, I ran a blog called Donkeylicious with my smart mathy friend Nick Beaudrot. As per the name, it was a Democratic-aligned politics and policy blog. Probably its most intense period of activity was during the legislative drama leading up to the passage of Obamacare.

Back during the 2008 election, I made a blog called "War or Car" about cool stuff you could buy for the $3 trillion price of the Iraq War. The name of the blog came from the fact that you could buy every household a Toyota Prius, but I usually explored more wacky things. For example, you could buy every panda a stealth bomber!  

For a brief period before Obama was elected, I was guestblogging for Kevin Drum of the Washington Monthly. It was a wonderful opportunity (thanks Kevin!) which I mostly used to say nice things about Nancy Pelosi and John Edwards. I feel much better about the Pelosi praise than the Edwards praise nowadays.

There are some old posts from me at Daily Kos, but those display me in a relatively primitive state of political understanding and/or John Edwards boosterism. Consider them added for completeness.