I Give Scores A 2 Out of 10
It’s a funny industry we live in. It’s relatively young and very foolish at times. The part of it I’ve spent lots of time with recently is what some would refer to as the ‘journalism’. On occasion I despair at it; I talk at people on Twitter about it and we all agree there are some things that just aren’t right (*cough*DoritosandMountainDew*cough*). Pardon me. Anyway, there’s one part of it all that’s a little less black and white and that is the use of review scores and the way we use them. Now, I’m sure this article won’t rock the gaming world, but it would be nice if TheGameJar’s little corner of it could have a chat about them.
I’m sure you’ve all seen charts like the one below for much larger sites and granted there are, at the time of writing, only a total of 44 game reviews on this site (there are two others for hardware but I’ve discounted them – if you were wondering they were both 9s) and there will be a couple more up by the time this goes out. At TheGameJar we like to think we give you honest opinions and have a fair, even score system. The first I know to be true but this graph proves the second otherwise. Now, the way I rationalise this is that there are multiple writers, all playing games and have enjoyed them, therefore have given a score they think appropriate. That makes sense to me because once a game has been played through and you think it deserves a 7 you’re not going to consider averaging that next to all the other genres and games that are on the site.
On the other hand, mathematics does say that the biggest spike should be in the middle with decreasing sized bars either side of it. Sometimes I find it difficult to give a score and spend far too much time thinking about that, when I could be writing something interesting about it for you guys!
When we talk about our reviews with other people occasionally they ‘strongly disagree’. Occasionally I have seen that no progress is made – everyone gets frustrated with each other. For example, here is a review of a building (because shut up):
“Battersea Power Station is a reminder that utility buildings can be functional and look good equally. It is a national treasure and after being featured on the cover of Pink Floyd’s album Animals it is even more so. Unfortunately after being decommissioned it has fallen into disrepair and is surrounded by rubble and broken windows. I give this building a 9/10.”
Without the context of the paragraph that came before it, the score has almost no meaning. Even with the rest of the review the score doesn’t add much because I have already learned what the person thinks of the building from the content and tone of their article.
Another problem arises from this. When a person just reads the score and thinks that Battersea is more of a 7, we haven’t learned anything from our difference of opinion – we are just arguing about numbers. I would be far more interested to know that they prefer Bankside Power Station down the Thames because it is less dilapidated.
They can be helpful as a buyer’s guide (deciding between two similar new titles etc)
It can be a good way of deciding how close the reviewer’s thoughts are to yours quickly
By making a chart like the one above we can see how even a site’s scores fair
They can get in the way of good discussion if somebody disagrees
They can be abused in many different ways (eg being paid for scores/user score bombing)
A user might just read the score and ignore the article
Sometimes I find it difficult to give a score and spend far too much time thinking about that when I could be writing something interesting about it for you guys! I rarely read a score when going to another site for their opinion.
For me scores are there to give the Dev / Publisher / PR Team something to benchmark against. The don’t care what is actually written, they just care about the scores and the fact that “7″ seems to be the new average (which is ridiculous) plays into their hands.
I’m more than comfortable with our scoring methods here at The Game Jar. A average game that has some good point and bad points in equal measure will get a 5. As soon as there are more good points you go up, more bad points you go down.
With so many publications, online and off there will never be a standardised scoring system which is what the industry as a whole needs if they are to stick with reviewing games by numbers. I would personally rather end up with just a written review with pros and cons and leave it at that, but a huge majority of the audience I imagine just come to see if their favorite game got a 10 (Ryan’s PES 2013 review for example).
If the scores are for publishers etc. then why do we put them up? I’m sure Jenn doesn’t intend for the site to be ready just by people in the industry. Even, then it is probably worth a developer’s time to read a whole article.
I know that we say that 5 is average but the chart says otherwise. If we want to continue with scores then it might be worth rethinking how we give them out and how to make sure that 5 is the average it is supposed to be (ie. the one with the most games attributed to it).
Its true that there will never be a standard (unless you count Metacritic, lol). I’m looking at just The Game Jar for this so we can be in the position we want to be regarding scores. What we should ask is do we want an audience that doesn’t read what we have to say and just looks at a number?
If you would prefer just having the article with some pros and cons then maybe it is time to think about a change.
Review scores are pretty much meaningless to me now, and it’s all because of Dragon Age: Origins. I bought that game without reading a review and loved it, despite it’s faults. After I’d finished playing it, I read a review that gave it a 6 and concluded it was an average game with some glaring technical faults. I thought about what they had written and agreed with a lot of their criticism, but in the end I didn’t care, I really enjoyed the game.
After that, review scores didn’t matter to me any more.
Good point. Do you buy games without reading reviews often? Do you think that is what many other people do?
That depends on the game I think. Will gamers who’ve bought every single Call of Duty since CoD4 read a review to determine whether they’ll be buying Black Ops 2? I don’t think so. They might read it to get a little extra info, but it’s not going to influence their choice to buy, they’ve already made their minds up.
In a way, games like CoD or Fifa are almost totally immune to reviews scores. They have a core audience that’s always going to buy the game, so what do scores matter, especially when a 7 seems to be the new average? Will moving BlOp 2′s score from a 9 to a 10 have any significant impact on sales? Probably not. Will moving it from a 10 to a 9 significantly damage sales? Again, probably not.
Unless it’s a really bad score, the number is irrelevant for AAA Ip’s in my opinion.
Also a good point! Perhaps you should have written this article. Outside of those core audiences is probably where the use for a review lies, but as you say that kind of audience won’t respond much differently to one point difference.
If a score makes no difference then why have one? Would it be work putting a score to a review case by case?
I’d quite like the removal of scores. The concept of an average rated game doesn’t really pan out in my head. I mean, an “average” game implies to me a game that is halfway between the best games ever (10 doesn’t equal perfect in my book) and the worst games ever. But what exactly *are* the worst games ever,what scores a 1? Or a 0? What is the average game being compared to? I mean, the word “average” implies a middle ground and as this site has not scored a game lower than a 4, a score of “average” kind of loses a little context . Also, as the Batersea power station example illustrates, a score can mean different things.
I don’t think scores add anything useful to this or any videogame site. In fact, in my humble opinion, the removal of scores adds credibility to a site or publication as all the meat of the review is in the writing. Some reviews on other sites can be so poor that without a score to lean on, the review would be obviously insufficient.
So, in the worst (very worst) cases, I believe scores can be a crutch for a lacklustre ludophile to lean on. They add nothing but confusion, a relic ready to be forgotten maybe?
Agreed review scores are perhaps better off as a relic to the past….I very rarely look at scores, prefering instead to just read the reviews. Chris is right as well, if I really like the look of a game I’ll buy it no matter what anyone says…perfect example is Deadly Premonition. A game which has become something of a cult classic however some reviewers really hated it. If I’d listened to them I would have prob missed out on a game i really liked