#1
|
|||
|
|||
Rating systems: 5 stars vs. something else?
I've been reading a bit about rating systems lately.
Apparently 5-star rating systems are not good, since most people rate really high or really low. YouTube used to have a 5-star rating system and "the number of 5-star ratings greatly outnumbered all others, with 1-star a distant second." People also use half-stars a lot less than whole stars. On some site called MovieLens with millions of ratings "only 20% of all ratings are half-stars." Link here: A Better Way to Rate Films ________________ I found a very simple, but effective way of using ratings. Assign a verbal meaning to each point in the rating system. This is the 10-point system that RPGnet uses: 10 - One of the Best Ever 9 - Outstanding 8 - Very Good 7 - Good 6 - Above Average 5 - Average 4 - Almost Average 3 - Some flaws 2 - Poor 1 - Worthless (I use this now for rating my songs on iTunes (with half stars) and my mood affects the ratings less, which is good. Ratings 3 and 2 to me are "bad" and "very bad," respectively. Sadly I have less 5-star tracks than before.) Link here: Collective Choice: Experimenting with Ratings ________________ This is also worth reading: Is there a better alternative to the 5-star rating system? Many insightful answers with lots of examples. ________________ How do you go about ratings albums on VGMdb, movies on IMDB or anything else? Are you happy with VGMdb's 5-star rating system and how people use it? |
#2
|
||||
|
||||
already discussed on the irc chan, and maybe on forums?
edit: I mean, it's been discussed and blah agreed about the fact that the actual ranking system isn't the best maybe a system rating track by track would be better but the problem would be how to implement that simply Last edited by Phonograph; Feb 23, 2013 at 12:39 PM. |
#3
|
|||
|
|||
VGMdb has a 10 point system. Half stars are functionally equivalent in all but graphical representation to points. I don't think about it differently here than at IMDb.
In my mind: 10 (5) - Best of the best 9 (4.5) - Excellent 8 (4) - Great 7 (3.5) - Good 6 (3) - Decent 5 (2.5) - Okay 4 (2) - Mediocre 3 (1.5) - Poor 2 (1) - Terrible 1 (0.5) - Absolutely horrible Quote:
Last edited by Xenofan 29A; Feb 23, 2013 at 12:31 PM. |
#4
|
||||
|
||||
Three star rating!
2 stars: great 1 star: good -1 star: no good Less choices = easier to understand! |
#5
|
||||
|
||||
the thing I hate the most about the 5 star system is that you can't see all ratings, not necessarily who rates, but the rates themselves
because if you have 3 people with 3 stars, you can't know if 3 people have rated 3 stars or 1/3/5 stars (already spoke about it) there should a rating page like there is a history page (for edits) |
#6
|
||||
|
||||
Yup it's like someone gives a 5 to a jpop single and then I come by and give it like 1.5-2 heh
But it's not about 5/10 stars there, just a rating history is missing. |
#7
|
|||
|
|||
Quote:
I didn't start this thread to change the rating system of VGMdb. I'm just interested in the subject in general. But if it's a viable option, of course we can think about changes to VGMdb's rating system too. I like this. Do you use this approach anywhere? Is it difficult not to have 0 stars or some option for "average"? Last edited by Namorbia; Feb 24, 2013 at 04:36 AM. |
#8
|
||||
|
||||
Instead ratings I personally prefer a relative ranking system purely by order. Usually I know pretty well myself what I prefer over other works at any given time, the issue is always how much value I attach to that (naturally heavily skewed to pushing the max, after all you want others to notice what you like...). Don't know how others feel about this, but I think it would be great to allow everyone to make personal ordered hit lists without any ratings attached (the order IS the rating), and the final rating value is then calculated based on the relative ranking a work has on those hit lists.
|
#9
|
||||
|
||||
Quote:
It's better if you don't have too many choices (think some movie reviewers), and "verbal" ratings like you said. So maybe - "(no vote) / lame / average / sweet / AWESOME". |
#10
|
||||
|
||||
Perhaps disposing of the rating system altogether in favor of posting reviews (likes/dislikes) or thoughts about the album on an entry's thread would eradicate unreasonably low scores that could deter a potential listener from an unjustly muddied, but attractive album.
|
#11
|
||||
|
||||
GameFAQs has descriptions next to its ratings for games, and I always agreed with what they describe as a certain score (out of 10).
Spoiler:
I've always thought these words are pretty basic quality descriptions, nothing fancy like "awesome" vs. "amazing" (which one's really better?), and they're distanced enough from each other so that there's really no confusion in what you think the rating should be if you go by the descriptions. I don't think 10 is too many choices. If I'm going to give something a rating, I want some room to distinguish it from things I don't think are quite at its level. For example, I think X is a 3.0, Y is a 3.5, and Z is a 4.0. These are very different rankings, because as it is, there will be things in the 3.0 ranking I like more than others in the same 3.0 ranking, but yet still less than anything in 3.5 - it's general but descriptive. But if we start to smush rankings into each other, now X and Y are in the same class, 3.0, and Z is way better than Y, 4.0. I like 1-10 ranking systems for this. Sure, we could have 1-20, 1-100, whatever, but 1-10 is pretty common and I think it works well. I don't think abuse is something anybody should try too much to combat, because it's just going to exist no matter what. Some people rate things 5/5 before the album has even come out. And some people rate 5/5 if they "like" it, so a good 80-plus percent of their ratings are 5/5. That's okay, let them rate how they want, and the people who want to make meticulous use of it will. I think it will be like that regardless of any system used. Wow... I need to get on it and give some more soundtracks ratings! And fix some of my old ones... they're from when I first joined and didn't really have enough sense of good ratings yet. Datschge: Sometimes I can easily say "I like X soundtrack more than Y", and then that would work. But sometimes the soundtracks are too different, and I like them "equally" for different things (and I quote "equally" like that because it's too hard even to say that since they're so different, they can't really be compared). If X and Y are really different from each other, but both deserving of a 3.5, I wouldn't want to have to rank them against each other, but instead I'd want to give them a rating based on what they deserve for what they do (in my opinion of course). Last edited by Mortavia; Feb 25, 2013 at 10:45 PM. |
#12
|
||||
|
||||
Quote:
Last edited by Datschge; Feb 26, 2013 at 03:26 AM. |
#13
|
|||
|
|||
I think we should just be like Facebook and have a Like button. Then you can just go around and Like all the albums you like and then there's no worries about whether or not somebody gave a fair rating. Plus then we're more like Facebook! I Like Facebook.
|
#14
|
|||
|
|||
This is actually pretty similar to what I use for VGM ratings. It works really well, especially since my two main goals when rating are to eliminate waste and identify the best tracks on a given soundtrack. So it's basically Pass/Fail with an extra step to give a boost to the really special tracks.
|
#15
|
||||
|
||||
actually, instead of changing the current rating system for a better one now, this one could be tweaked a bit
removing the current average thing and limiting to 5 stars or even 3 as follows *****_(excellent?) nb of votes ****__(good?) nb of votes ***___(ok?) nb of votes **____(bad?) nb of votes *_____(forget?) nb of votes edit: I mean showing those 5/3 (perso, 3 is enough) lines, in case it's not clear enough Last edited by Phonograph; Mar 7, 2013 at 06:47 PM. |
#16
|
|||
|
|||
I removed half-star ratings from iTunes. It takes too much thinking and looks ugly. Just look at a list of tracks like this:
***½ ****½ **½ ***** ***½ **** *½ ***½ It looks confusing and I can't quickly pick out the best tracks (unless I order them by rating). I replaced my thinking like this: 10/9 -> 5 (best ever OR outstanding) 8 -> 4 (great) 7 -> 3 (good) 6/5 -> 2 (average) 4/3/2/1 -> 1 (bad) The list of stars above would now look like this: *** ***** ** ***** *** **** * *** My point is that "average/ok" should not be in the middle. If a track is bad, I don't care if it's very bad or not. That's why "average/ok" should be 2 stars in a 5-star or 4-star rating system. |
#17
|
|||
|
|||
See, that is the issue I started having...Average tracks should not be rewarded. They just take up too much room when they come up with very little reward. But it makes sense, rarely do I play a full game where every piece grabs my attention. So I decided to start removing them, as I would a truly "bad" track.
I think on average, when I lay down a rating on a full soundtrack, I usually end up with a near perfect split: 1/3 bad/lackluster/filler tracks, 1/3 "average" tracks and 1/3 "really good/great" tracks, which really should be the focus of any future listening I do. |
#18
|
||||
|
||||
Quote:
average of 1+2+3+4+5 is 3, not 2 you should simply use another word, average at 2 is confusing |
#19
|
|||
|
|||
I don't think an "average" track is the same as a mathematical average. If anything, it's weighted much lower (calling a track "average" is almost never a compliment).
|
#20
|
|||
|
|||
Here are the ratings I have for Seiken Densetsu 3's first and second discs. Some bias exists (it's one of my favorite soundtracks), but that top row makes it easy to pluck out my favorites quickly. The ones on the bottom row, I delete from my computer, while the ones on the middle row, I tend not to carry them around on my portable players.
Disc 1 P! - 2, 3, 5, 6, 7, 12, 13, 15, 16, 18 P - 4, 8, 14 F -1, 9, 10, 11, 17, 19 Disc 2 P! - 7, 8, 10, 12, 14, 19, 20 P -1, 11, 13, 21 F- 2, 3, 4, 5, 6, 9, 15, 16, 17, 18 |
#21
|
|||
|
|||
Wow, you listed some of my favorite tracks from SD3 as filler! Different tastes, I guess.
|
#22
|
|||
|
|||
Yeah, I tended not to favor a lot of the darker/more atmospheric tracks over the more upbeat, action-oriented tunes. Then again, I knew before I sat down to rate, I was never big on the first half of disc 2.
|
#23
|
||||
|
||||
Wow, I am suprised. I mean even taking out the dark/atmospheric/somber tracks you've still got Another Winter, Harvest November and Few Paths Forbidden (1.9, 2.3, 2.4 resp) in there. Granted not the upbeat stuff, but hardly dark and definitely colorful and pure Kikuta!
Last edited by yvk2000; Mar 8, 2013 at 12:30 PM. |
#24
|
||||
|
||||
In the end it's useless to discuss about differences in personal taste.
I don't like almost all of j-pop, but when somebody praises a certain j-pop song/single/album/whatever, I won't start saying dude wtf and similar (just a random example) To each his own. |
#25
|
|||
|
|||
Sometimes it can be interesting to discuss taste. There are things most people agree are bad and things many people agree are good. You can try to understand why others appreciate something that you don't.
|
#26
|
|||
|
|||
Darn it, I'm missing out on this thread being derailed! This has become about discussing personal taste? Oh, let me in on this! I love to discuss personal taste. I personally think that entire soundtrack sucks! There was not one song on it that I liked. That is my personal taste.
|
#27
|
|||
|
|||
Nah I'm just kidding. I actually liked every single song except for one. I don't know how you guys could dislike any song on that soundtrack (except for that one).
P.S. Efendija is right. It is the most utterly stupid thing EVER to discuss personal taste about art. Example 1: "I really like this thing!" "So do I!" "Okay, cool!" Example 2: "I really like this thing!" "I don't, it sucks!" "Really? Wow..." Who cares? And you can't "understand" why somebody likes something. That like trying to "understand" why I like broccoli while you think it's the most horrible tasting thing you've ever eaten. Me: "I think broccoli tastes really good because ______!" You: "That's weird, I don't!" NOT: "Oh, okay, I get it, now it tastes good to me too!" Discussing taste = fail and this thread is about ratings systems, seriously. I vote a Facebook Like system guys. Who's with me?! Last edited by Hellacia; Mar 8, 2013 at 01:31 PM. |
#28
|
||||
|
||||
I'm not. I hate Facebook. And a simple Like-system is nothing more than what's currently the amount of votes albums get disregarding the actual rating, which would only make popular works stick out even more and less known works never see the light that way.
|
#29
|
|||
|
|||
My rating system was a demonstration of showing which tracks I liked and disliked from the subject matter in basically a capsule form. Based on the follow-up and said follow-up nearly derailing the entire thread, with basically no additional input from myself, I'd call it a rousing success!
(well, I already know it is one, since I have hundreds of those little P!/P/F grids for various discs/albums, but it's the first time I've used it in public.) Free bonus: SD Disc 3 P! - 1, 3, 4, 5, 7, 8, 14, 15, 17 P - 2, 9, 12, 13, 16, 19, 20 F - 6, 10, 11, 18 |
#30
|
|||
|
|||
Quote:
This is a question for everyone: Do "meh", "okay" and "average" mean the same thing and have the same rating? Or is it more like okay>average>meh? How many listens do you give to a track before rating it and deleting it? Maybe you will start liking and appreciating the bad tracks after a few more listens? Ever regretted deleting tracks from your computer? This brings me to the next quote, which I disagree with: Others opinions of a track, especially positive opinions, have an effect on me. When a friend points out what's good about a track, I often start liking it more. This effect can be achieved by listening to VGM podcasts also. Maybe it's simply taking the time to really listen, be attentive and realizing the good parts. The more friends I have who like VGM and point out the good parts, the broader my taste becomes! Last edited by Namorbia; Mar 9, 2013 at 12:28 AM. |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Dysfunctional Systems: Orientation Soundtrack | ff80c38 | Album Discussions | 0 | Jul 25, 2017 09:48 AM |
ARCD0001: The regret of stars, but stars shine bright. | Faalagorn | Album Discussions | 2 | Apr 9, 2011 05:09 PM |
Missing systems | dma | Questions and Comments | 4 | Mar 15, 2010 03:42 PM |
Album rating function | Gaia | Questions and Comments | 6 | Oct 30, 2008 07:10 AM |