The Rhomal Report - Skewed reviews?

[Rhomal]

As you may have seen a few days ago I posted an Op Ed article from the Inquirer about the possible implosion coming up in the console market. But one aspect, that affects all gaming, stuck out in my mind. And that was this part of the article:

 

"There is one way out though, read reviews before you shell out a day's wages for a potential stinker, read anything you can on the game. Remember when I said in the old days there was a co-opting of magazines, that was in the pre-web 1980s? Well, now it is worse. The pestilence of flash aside, most gaming sites and magazines are so far from publishing a real review it is laughable.

 

If you want a good example of this, take the new Madden game. Go to the major sites and look at the scores they gave it. Read the reviews, and compare that to the numerical scores. Then look at sites that put reader ratings next to the official scores. Notice a discrepancy? For bonus points, find me a major ratings site that regularly gives out a score of 6/10 or less. Once again, they are owned"

 

This got me thinking, looking at the major online PC gaming sites, I am hard pressed to recall more then a couple games within the recent past months that had a below average rating. Now unless we are going though a new golden age of PC gaming shouldn

[Judge Hades]

That is a very good article, Rhomal, and you come up with some good points. What we need is a gaming site that is very very harsh. Eurogamer was damn harsh on PD0 for the X Box 360. We need more reviews like that on a regular basis.

[Volourn]

The thing to remember is that video games aren't rated on a 1-10 or 1-100 scale for the most part. They're rated basically on a 6-10 or 60-100 scale. Remember that, and for the most part they make sense.

 

Besdies, as for reviews, it's best to pay attention to what the actual review states about the game; not the number.

 

Afterall, GS gave JE 84% but it's apparant to everyone that the reviewer didn't really like the game just by what and how he wrote it.

 

 

[Judge Hades]

I gauge games from a 0 to 100 basis then assign a grade letter to them. Such as I gave NWN a 75% which is a C grade. LotOR deserves a B+ so I gave it a score of 90%.

Edited December 1, 2005 by Hades_One

[Volourn]

75% is a B.

 

More seriously, I try to as well; but the review sites don't. So, when we see a game get a 60% that's a low end for them so we have to keep that in mind. 70% = ok or average, 80% is good, and 90% (as per usual) equals amongst the best.

 

I'm not saying I agree with the thought process; but it's soemthing to keep in mind. Nothing to worry about, however.

[Judge Hades]

75% is a B.

 

Only if you ride the short bus or from Kanada. :ph34r:

[Volourn]

Isn't that the same thing? :D

[Sarjahurmaaja.]

I don't rate games on a numerical scale. It's stupid.

 

Also, I'm shocked and surprised. Shocked and surprised!

[ShadowPaladin V1.0]

Most sites dont use the full scale.

 

If you read the review you can usually tell what the game was marked down for and see if that applies to you.

 

For example if a game scores 60% and the reviewer has a rant about how there is no functional multiplayer. If you dont care about multiplayer , then it wont be a 60% to you.

 

NoM has a good system they have a list of what the positives and negatives are , thats easier than having to pick them out of the review.

[Jumjalum]

Keep 'em high I say, I want to see classic after classic when I visit a review site.

[Volourn]

"reviewer has a rant about how there is no functional multiplayer. If you dont care about multiplayer , then it wont be a 60% to you."

 

Exactly.

[Dhruin]

Your concept is somewhat flawed, Rhomal. I don't write this to defend the major sites but to debate the article.

 

Your premise is predicated on each of these sites having reviewed every game that came out for the period, across all categories. But they don't. They don't review many indies, smaller games or Euro titles - this automatically removes the bottom tier and leaves you with professional titles that managed to secure a US publishing deal; most of the real crap is already eliminated.

 

It would also be nice to know what period you counted. The RPG market might be crap but frankly, there are some pretty good games out there in other categories.

 

I don't know anything about Madden - don't play sports games. But... is this it?

 

http://www.gamespot.com/xbox360/sports/mad...w.html?q=madden

 

GameSpot review - 7.4, average reader rating, 8.3 across nearly 700 votes. If that's it, you'll need a better example, no?

[ShadowPaladin V1.0]

Heres the important bit.

 

The Bad: Many of the unique gameplay features of the last couple of Madden games are missing; lots of missing modes--all you get is a husk of a franchise mode and online play; some unsightly animation and graphical hang-ups that stick out all the more on a next-gen console; you can't challenge plays.

 

Probably enough there to justify the score , which is only going to be approximate anyway since it's one guys opinon.

 

Wouldnt place any faith (actually less) in the reader votes since people go around inflating and deflating the scores.

[Dhruin]

I'm not equipped to discuss Madden but it seemed to be Rhomal's specific example. Every reviewer makes mistakes and unquestionably some sites are better than others; I do particularly like Eurogamer, personally. I just don't think showing that a certain percentage of scores above an arbitrary cutoff proves much, without showing a consistent theme of individual reviews that have been inflated - and I'm not sure that can be easily done. I never bother with user scores but people often point to Gamerankings or similar, so plenty of other people put stock in them.

[mkreku]

We've had this discussion on Spel

[ShadowPaladin V1.0]

I thought KONG was seriously repetative. Once you got the whole "Wow I'm a giant ape that can wrestle a TRex" out of the way.

 

If your going to go for an "objective" review then the best thing to do is list the reasons you marked the game down and let the individual reader determine how important those factors are to him.

[Slowtrain]

Reviews are all opinion anyway, aren't they?

 

I mean does anybody seriously buy or not buy a game based on what a couple random people, whom one doesn't know and will never meet, write about it?

 

That's like me not buying Arcanum cause SP thinks it is a giant piece of poop!

 

 

And that aIn't gonna happen.

[Rhomal]

> Your premise is predicated on each of these sites having reviewed every game that came out for the period, across all categories. But they don't. They don't review many indies, smaller games or Euro titles - this automatically removes the bottom tier and leaves you with professional titles that managed to secure a US publishing deal; most of the real crap is already eliminated.

 

So your stance is every or most AAA titles are a 80%+ game and every indie game is bottom of the barrel by default?

 

> It would also be nice to know what period you counted.

 

Sorry, I dont understand what you are asking.

 

> I don't know anything about Madden - don't play sports games. But... is this it?

 

If you are refering to the op ed from the INQ I quoted, you will have to ask that editor about the hows and whys of his article. I just quoted the part of his article to show what got the gears going in my brain on this topic for me. The source of my interest to explore this topic of you will.

[ShadowPaladin V1.0]

Reviews are all opinion anyway, aren't they?

 

I mean does anybody seriously buy or not buy a game based on what a couple random people, whom one doesn't know and will never meet, write about it?

 

That's like me not buying Arcanum cause SP thinks it is a giant piece of poop!   

 

 

And that aIn't gonna happen. 

<{POST_SNAPBACK}>

 

Yes but it's an opinion that should be grounded in some sort of evidence. IE Arcamun is crap because it's full of serious bugs - yes thats based on evidence. Arcanum is crap because its poorly balanced - again you wouldnt find much argument there.

 

Well that would be you overlooking the flaws rather than because the game dosnt have them. A reviewer that didnt bring those flaws to attention in the review would certainly not be doing their job.

[LoneWolf16]

I haven't played a truly "Bad" game in a long, long time.

 

Every title I've picked up has had it's merits (neat features, great game play, enthralling story-line, well written dialogue, masterful music, realistic ambience, pretty graphics, exciting plot twists, excellent voice overs, etc.) as well as its flaws (bugs, bugs, more bugs, dissapointing endings, crappy game play, writing similar to that of dyslexic ape on crystal meth, poor support, bad developer decisions, etc.), yet none I've seen have warranted a score, from me, below a 6/10.

 

I may just be generous, but I haven't stumbled onto a truly awful game that didn't get a truly awful review.

[ShadowPaladin V1.0]

It was pretty funny that they pulled the 2/10 Fifa review from Eurogamer.

[Hurlshort]

I only trust Computer Gaming World when it comes to reviews. Even so, different reviewers have different tastes. Also, Xbox360 games will probably be given a bit of a boost in reviews because the system is new, they have fancy graphics, and there isn't much to compare them to.

 

I've also noticed how little reviews have to do with sales figures. Madden doesn't need good reviews. It will just sell like crazy.

[alanschu]

I gauge games from a 0 to 100 basis then assign a grade letter to them.  Such as I gave NWN a 75% which is a C grade.  LotOR deserves a B+ so I gave it a score of 90%.

<{POST_SNAPBACK}>

 

I've always wondered where you magically pull these numbers out of?

[Slowtrain]

Yes but it's an opinion that should be grounded in some sort of evidence. IE Arcamun is crap because it's full of serious bugs - yes thats based on evidence. Arcanum is crap because its poorly balanced - again you wouldnt find much argument there.

 

Well that would be you overlooking the flaws rather than because the game dosnt have them. A reviewer that didnt bring those flaws to attention in the review would certainly not be doing their job.

<{POST_SNAPBACK}>

 

But, while I agree that Arcanum was bug-heavy and quite unbalanced still even after the final patch, there are so so so many awesome things in the game that I don't even notice the bugs really.

 

SO while a reviewer may be doing the job by bringing up notice that a game is buggy that still doesn't mean that I won't enjoy the game.

 

Again, in today's gaming market, you simply assume that a game is going to be buggy upon initial release, so unless thay are system-destroying apocalyptic bugs, its not terribly useful info that a game is buggy.

 

edit: spelling. bah!

Edited December 1, 2005 by CrashGirl

[Slowtrain]

I've always wondered where you magically pull these numbers out of?

<{POST_SNAPBACK}>

 

 

I could tell you, but then I would be banned. :cool: