Views from Medina Road

As I've threatened to post several times over the past couple years, here's some statistical commentary on voting patterns in the annual Locus polls, using completed results from the past two years as data samples.

What has always struck me in looking at the ballots as they come in, and as they're compiled and tallied in the database, is how few voters submit a complete or even mostly complete ballot. Consider that there are 14 or 15 categories -- 4 for novels (sf, fantasy, first, YA), 3 for short fiction, 3 or 4 for other books (collection, anthology, nonfiction, and in some years a separate art book category), plus magazine, publisher, editor, and artist. That's 70 or 75 nominations a voter can make.

In 2004, there were 14 categories for a potential 70 nominations per voter. Of 629 valid ballots, only 29, or 4.6%, filled out every category. At the other extreme, 19 voters submitted ballots without any votes at all. They did fill out the survey, and all but 2 indicated they were subscribers, so perhaps they just wanted the free issue that comes with submitting the survey.

The average number of nominations per voter in 2004 was 29.8, with a standard deviation of 19.5.

In 2005, there were 15 categories, and 913 valid ballots. Of them, 55 voters, 6%, filled out every category, and 20 didn't vote for anything at all. The average was 30.2 votes, with a standard deviation of 22.

The most noticeable difference from 2004 to 2005 was the number of online voters, including nonsubscribers. That trend seems to be increasing in 2006, with over 600 ballots received in the first month of voting and 6 more weeks to go.

Which categories are most popular with voters? Here are the numbers, where the counts of voters are the number who nominated at least one item in the category.

category	2004 voters	2004%	2005 voters	2005%
total	629		913
SF novel	538	86	756	83
Fantasy novel	514	82	768	84
YA novel	346	55	558	61
First novel	408	65	660	72
Novella	360	57	511	56
Novelette	374	59	503	55
Short story	356	57	519	57
Magazine	453	72	639	70
Publisher	502	80	699	77
Anthology	440	70	544	60
Collection	413	66	563	62
Editor	402	64	595	65
Artist	330	52	540	59
Monfiction	351	56	408	45
Art book			390	43

Unsurprising conclusion: More people vote for novels, with magazine and publisher categories also very popular, while short fiction, artist, and nonfiction book categories are least popular. I would guess without checking that these trends are similar to Hugo voting patterns.

It's also interesting to quantify the results of the voting in terms of the number of voters. To an extent you can observe this from the published results of the poll in Locus Magazine, which give the total number of ballots received each year and show the number of votes each item received (as well as number of 1st place votes, and total number of points). Consider that the winning novel in 2005, Neal Stephenson's Baroque Cycle pair, attracted 219 votes (97 of them first place votes, for a total of 1503 points). That's 219 votes out of 913 voters, just 24%. It's only 29% of those who voted in the category at all, and keep in mind each voter has 5 nominations per category.

Even a perhaps more generally popular book, Susanna Clarke's Jonathan Strange & Mr Norrell, attracted just 426 votes (in the first novel category), i.e. votes from 47% of all voters, 65% of those who voted in the category at all.

Clarke's results are exceptional; much more typical for novels and short works are numbers in the 20-25% range. In 2004 the most-voted fiction work was the winning SF novel, Dan Simmons' Ilium, with 198 votes from 629 voters (31%) of whom 538 voted in the category (37%). In contrast, the winning novelette that year, Neil Gaiman's "A Study in Emerald", got just 69 votes from the 374 voters who nominated in the category--18%.

The point is that it's extremely rare for an award winning book or story to actually by favored by a majority of the voting population. Certainly there are extenuating circumstances; some books take a while to become well-known, while many readers don't get around to books until they're in paperback. But as a voting result, I suspect this has always been true, even for 'classic' works that are now taken for granted as the most popular works of all time. Even Dune tied for a Hugo in its year!