RF Generation Message Board

i like most of what i see with the site, but i have one beef

the rating system

a 100 point or even a 10 point scale is far too large to be used effectively without specific criteria for a game to earn points.  in other words, what is the true difference between a game with a score of 87 versus a game with a score of 86?  what about the difference between a 2 and 3 on a 10 point scale?  its far too subjective and arbitrary, and in most cases reviewers will just poop out a number only loosely relative to other scores.

if you want a true objective "top 100" and "bottom 25" list, you need to remove as much subjectivity from the scoring process as possible.  for years, roger ebert has rated films with a simple "thumbs up" or "thumbs down".  this may be a tad too draconian to construct an objective greatest games list, but with a little tinkering it can be fixed.

a three point scale (bad/average/good) can rate over 90% of the games on the market.  for the 10 percent of games that are true classics or horrid pieces of turd pudding, you can add two extreme points on either end of the scale.  what you end up with is a well defined 5 point scale.  and over the course of 100s and 1000s of reviews, you can get a very accurate ranking of where a game stands within a community.  

look at imdb as an example of a 10 point system gone awry,  there is competition at the very top and the very bottom for voters to stuff the boxes with very high and very low scores to adjust the position of their favorite best and worst movies.  a five point scale can reduce the effectiveness of ballot stuffing

A few things on this subject. I truely think if we had everybody vote on the games they own, and eventually when we have thousands of members, the 100 point scale would truely be worth it.

A 5 point scale to me is too narrow IMO. A "3" rating could represt a fun semi good game or a below average crap game good enough to make a passing grade. 3 = to 60% if broken down

5, I would asume would be equivelent to 90% or higher? I want to know if it is truley kick ass 95 or higher and not just eeking out a 90% to earn that 5 rating.

4, Would be your 70 - 90? There is just too many titles you can throw in this category.

I know X play uses a 5 point scale. There just isn't enough detail in a 5 point scale (IMO)

I know 1 - 100% is a bit much, but maybe a 1-10?

Either way, I see your point and hopefully this turns into a nice debate over the issue.

i like the 1-10 system myself, it gives that much needed space between games, especially if you want to compare sequels or multiple platform as you've got enough lee-way to distinguish between a 9.5 game and it's 9.0 sequel or a 8.0 PC game versus it's 7.5 console port that has a few bugs.

a ten point or 100 point scale is fine... assuming there is specific criteria for such scores

there isnt

i understand the want for such systems, as you mentioned, its easier to get a mental rank of titles when there is a broader range of scores to judge them, but with large scales, you get more subjective results and things like grade inflation, where average games are getting 7s and higher.

a five point system is very easy

you play a game

if it is average, you give it a 3 - most games should be in this range
if it is above average, you give it a 4
if it is below average, you give it a 2

often times, you will see extra little points added for things that dont really add to the gameplay of a game.  who cares if the face models on madden are slightly better or worse than last years edition?  details like these can be covered in a written review, ive never based a decision to buy a game based on these details, and too much focus on this stuff is one of the factors sapping creativity out of the industry.

a wide range 100 point scale is subjective, and applying an objective averaging system to a set, no matter how large, keeps the results subjective.  



a 3 represents an average game.  its not "bad", but it fails to hold your attention much longer after buying it



there is no such thing as a perfect game, even simple pong was revised and tinkered with several times.  a score of 5 is simply reserved for those games that can hold your attention for years.  fiver percent or less of games should get this score



4 is simply above average.  the game is fun, it holds your interest for a few weeks or months, and was a good value for the price paid.  20 percent of games should fall in this range



the details should remain in the meat of the reviews, not the score



i do too, and i want to see a reliable list of good games, but what makes barbie twice as good as cheetaman?  a brown turd and a green turd are both still turds

Like cap'n said, once lots of people actually start voting the 100 point should work fine. Right now, with three to four votes to a game, I don't think it matters what it is, it's going to be all screwy anyways. I do think 3 or 5 points would be too sparse, 10 would be fine probably, but hey 100 is fairly unique.

its still too subjective, all it takes is one person giving cheetaman a 100 or zelda a 2 to completely ruin the average.  

That's what I mean when I say no matter what it is it's still going to be all screwy. It's going to be subjective with any number of points until enough people are voting. Unless we used three or five, which I don't think works very well with a large community. 100 points gives a good, exact measure in my opinion. Yes, somebody could give an awesome game a low score but that's the problem with teh interweb.

it gives a good exact measure... but of what?  there is no real criteria on a 100 point scale.  you can never remove subjectivity from a judgement system, but you can reduce it by making the scale more narrow

Yes it could ruin the score now, but eventually when we have more members and  votes on these games, then it wont change the score much if at all.

Maybe we could eventually change it so a game wont appear on the list until after 10 votes.....

Really, any voting can be broken down. It's all relative.....

1 - 10 scale broken down into percentages.

1 = 0 - 19.9 %
2 = 20 - 29.9 %
3 = 30 - 39.9 %
4 = 40 - 49.9 %
5 = 50 - 59.9 %
6 = 60 - 69.9 %
7 = 70 - 79.9%
8 = 80 - 89.9 %
9 = 90 - 99.9 %
10 = 100 % perfect

1 - 5 scale broken down from 1- 10 scale

1.) score of 1 or 2 on scale of 1-10
2.) score of 3 or 4 on scale of 1-10
3.) score of 5 or 6 on scale of 1-10
4.) score of 7 or 8 on scale of 1-10
5.) score of 9 or 10 on scale of 1-10


So you can convert any scale back and forth.

when you get start to get hundreds of scores on a narrow scale, you will get a good specific number that is based on a far more objective system.  

i do not mind a wider scale, so long as the criteria of a scale is recognized by everyone... what is the difference between a 2, 3, and 4 on a ten point scale, really?

I like the 100% that we have. Like Cap'n said, we can get a better view of it all once we have a few thousand plus members.

im willing to bet if you look at a curve of all the ratings so far, the middle will not be in the 50s or 60s, but somewhere in the mid 70s.  leaving essentially a narrow range of scores for good games, and a gaping hole of scores for crappy games.  

Maybe we need a disclaimer / tutorial about how to properly rate games on this scale? I havent really looked into it as I rate my games properly (at least I think so ;))

exactly :)

Game Informer is a 1-10 scale yet 7 is average.

Once we get more members voting, it'll be really easy to get rid of those scores that mess up the ratings.  Like a real survey, drop the lowest 1% of the scores and the highest 1% of the scores.

That way:

If you have scores of 80, 76, 100, 2, 69.  Your average would be a 76.  

This will/would work a lot better on a larger scale.  The only problem with this method is that you'd almost never have a game rated 1, nor a game rated 100.  But no matter what scale....This is usually the case.

Ken

is that on purpose or an accident?

this isnt school, if you are going to have an effective scale, avergae games should be in middle of the scoring range.

It is on pupose. Over the years an average score for them is 7. I still think it is realy stupid. 1-10 average is 5 IMO.

with a narrow scale, you wouldnt need to bother removing outliers

No matter what the scale is, there's always going to be people ranking games the highest and lowest.  With a 5 point scale, people will be ranking 1s and 5s more than anything I would think.  The scale could be changed, but right now there just hasn't been enough people rating to justify that.  The way it would work is that the 100-point scale would stand, but as captn said earlier it would scale.  So a 1 would be 20, 2 would be 40, and so on for a five point scale.  So it would be the same scale, just not as precise as it is now.

Until people start rating games seriously, I really don't think a change is needed.

if each person's rating was listed with their collection then you could see the fanboyism and hate as a pattern, then it's either vote fairly or not at all because everyone can see them openly.

Just wait for people to start voting and it will even itself out like everyone says.  We just need more votes.  Plain and simple.

It is listed if you go to the Detailed view.

I'm gonna start rating more games soon as well, been holding off on that for a while now ;)