Episodes: Why the five-star rating system is the best rating system

When I got to Vox, one of the first questions I had for my new bosses was: Did they want ratings on our reviews?

The answer was yes (much as I had secretly hoped it would be no). And it's easy to see why. For more casual consumers of pop culture -- the sort who read Vox -- it's nice to have the quick way to get a read on just how a critic might have felt about something.

So the only question from there was what sort of rating system we wanted to use. And once that question was asked, I had only one answer, because it was the correct answer: five stars. (We made them five V's, and I always enjoy when people say that I gave something four stars getting to correct them by saying, "Actually, it was four V's," because I love pedantry more than my wife.)

This might sound silly, but I have put a shocking amount of thought into this over the years. A good rating system allows for enough room for nuance, but not so much nuance that readers are left scratching their heads. For instance, at the AV Club, the various grades up and down the line had such fine distinctions between them that it wasn't uncommon for readers to wonder just what the difference between, say, a B+ and an A- was, especially in TV Club pieces that tended to be written by folks who were willing to cut the shows they were covering a little slack. (We did have a very rough guideline, but it was inconsistently applied.)

I should note here that if you're certain only one person is going to be using the rating system, then you can probably get away with a lot more in the way of levels of gradation. Mike D'Angelo, for instance, rates every movie he's seen on a 100 point scale, and because it's just one guy doing it, I can sort of stack a movie that got a 68 up against a movie that got a 67 and see why one got one point more than the other, from his point of view. But when multiple people are going to be using the rating system, that inevitably breaks down. My 67 might be much better than Caroline Framke's 68. So, in essence, you're looking for something that's a little bit elastic -- able to accommodate lots of people.

For this reason, I don't really like the four-star scale. It doesn't have enough room to maneuver. The gaps between **.5 stars and *** stars and between *** stars and ***.5 stars end up being really, really vast, and a bunch of movies get classified as "***" (the line between "fresh" and "rotten," basically) in this system that are less liked than genuinely enjoyable movies with some big problems that get stuck with the *** label as well. In essence, the four-star rating scale lacks a true middle ground, and that keeps it from succeeding.

The five-star scale has just enough nuance. In the five-star scale, *** is a true middle ground, one that we've used a shocking amount of times at Vox. (We tend to only review things that we think are worth recommending or interesting to talk about, which skews us toward positive reviews, but I was still shocked when grabbing a little data for this that our number one ratings is ***.5, which is basically the equivalent of a B on the AVC scale.)

But it's also not so filled with ratings that they functionally become meaningless. One of the reasons we used D's so rarely at AVC, for instance, is that once you dropped below C-, then further gradations of "this is bad" became sort of pointless. You had D- for "bad and boring" and F for "bad but maybe entertainingly so if you're drunk enough." But D and D+? There wasn't a good reason for them. Compare this to the five-star scale, where the only arguably superfluous rating is .5 stars, which feels like you should probably just bite the bullet and go the full 0.

Plus, I mean, five stars is built around a base-10 system, and humanity has evolved to set things up in 10s from very young ages. If I tell you that one thing is worth a 7 and another thing is worth a 4, I think that's much more understandable than one thing is worth a 42 and another is worth a 38. You more intuitively grasp the difference without too much work.

(So why not go to a 10 point scale? Because the temptation is always to add in half-points, which effectively makes it a 20-point scale, and that way madness lies.)

I mean, yes, the best solution here would be to not have any ratings system at all. But that presumes some sort of utopia where readers are thoughtfully engaging with the substance of an argument, not just skipping down to comments to argue about how that movie was really more of a B+, instead of a B. And I think we all know which universe we live in.

(It's the latter.)

(Sidebar: The worst rating system is the San Francisco Chronicle's little man. Incidentally, Roger Ebert loved the little man. He wasn't right about everything, and he was dead wrong about this in particular.)

--

Episodes is published three-ish times per week, and more if I feel like it. It is mostly about television, except when it's not. Suggest topics for future installments via email or on Twitter. Read more of my work at Vox Dot Com.