Im not totally sure if the reviewers are doing this properly. Having equal importance on all those weird gategories makes this already so wrong.
And then you dont even think why the map is good or not, its more of a personal preference. Like in mineralz someone said the terrain is really plain...What would you expect. Its randomized and that adds a huge amount of replay value to the game. The game is simple and lacking some doodads really makes no difference in gameplay.
I suppose I just got a bit pissed since you didnt have proper statements to back up your scores. I personally couldnt even review sotis since I just played so much dota before that the sotis wont just do the trick anymore...
also whats up with the wierd review videos? watched 3 and they all had really different styles.
Cool idea but here's my thoughts (similar to zenx1 I guess):
Hard to judge maps of such different genre (under one number rating), right? Like me personally, I've made it well known that I can't stand turn-based text games. As good as Mafia is technically (and it's creator did a great job) I haven't enjoyed it and wouldn't enjoy it but that's just because I don't play those games... for the same reason I would probably not play Debates. Gotta be twitch-based!
So to me a 4.06 could be misleading? I don't know, I guess it's subjective.
Of course ratings are always going to be subjective, and I'm sure the rating system might have some flaws that will be worked out.
Regardless, I think having someone rating these maps in any capacity is really good for the community. Before SC2 was released, I thought the publishing system would also allow players to rate or review a map that's in the Blizzard database. That doesn't exist, and popularity is just a completely different metric from a real community opinion.
@SkrowFunk: Go I agree. Blizzard shown that HoTS will have a rating system, but anyway I'd like to see a more detailed system than simply giving the map 0 to 5 stars.
@Malpheus: Go Hey, who's in charge of picking maps and rating them? You guys should avoid the already popular ones and rate some unpopular maps (like mine) ;-)
Agree with the rest. I saw some of the Reviews and wanted to punch people in face for giving a 4/5 for some of the Terrain. -4/5 would be more like it for a lot of those. Just because the majority is bad, doesn't mean less bad is better in any respect.
Besides, rating systems are flawed in their very nature to begin with and it's probably better to give a First Impression like TotalBiscuit does.
Both magecraft and minerals have unfair disadvantages. Terrain. Neither of those maps can possibly have eye candy terrain. But the randomly generated arenas add alot of replayability. I think terrain should be replaced by appearance. Which would involve all eye candy material. Such as interface, effect and terrain. This would give games like magecraft and minerals a better balance in the rating.
We did try to substantiate the ratings, alot of content was lost (articles mainly) due to a recent host change and software change.
As far as videos are concerned, the majority of the videos were simply intended to give a gameplay overview, and the majority were not our videos, normally another reviewer or the mapper's video. I am still working on consistent video presentation.
Over the weekend or near to it, I will totally change the format to address the different genre's of maps. When we initially conceptualized the process, it was meant to reward creativity and fun, hence the high mafia rating. We have yet to weight the six categories or change them, input is certainly valued in that respect.
@SkrowFunk: Go
Ty for your support. We geared the reviews to address the map from the players perspective. Each of our reviews has at least one map maker and two players. I spent alot of time looking at discussions on bnet and here to determine this. The subjectivity of any review is absolutely unavoidable, essentially we are trying to figure out if the darn thing is fun. Instead of technically qualifying a rating, we use subjective statements from the reviewers to obtain a score. I would LOVE feedback on how to improve the accuracy of the five available statements within the six criteria.
It was started and halted till after your most recent big update, which has occurred. I'll put it after the two in que now. The most important thing I hope everyone grasps is we prioritize all requests. When there are no active requests, we simply troll to find the mapper making the most noise about a new map. If there isn't a prominent fuss being made over the latest, greatest, then we default to looking at the popularity list and up and coming list for interesting candidates.
I think splitting the maps into genre's will help with this, but ultimately there are six factors. The appearance of a map is not limited to terrain. If you were to look at "trifaction" terrain you would be unimpressed from the editor view, but if you look at it third person, it looks pretty. It really is very subjective, ultimately that is why we use three people at a minimum to offset extremely subjective standpoints.
And all that being said, the most important aspect of the process is the challenge. If it's not accurate by your determination, please say so. Specifically. Each map reviewed gets a challenge if someone feels they got a bad bounce. The best example of this would be WW2 diplo, we rated it unfairly, it was pointed out, the rating was removed and the new one is pending my third rater finishing his end up.
ALSO I WOULD ABSOLUTELY LOVE VOLUNTEERS FOR THIS PROCESS, please let me know :)
Updated to include the Element TD Rating. The hard part to rate on this one was the fact it's really a sequel of the WC3 map in the truest form. Unlike a remake like Squad where someone did a legacy import of the terrain and built from there, emulating anothers work, this time we have the original maker remaking his own map on SC2... hard indeed.
It faired much better than the legion td clone, but I still felt like i was playing Element TD on WC3 ultimately.
Up Next Fleet Assault and Starship Troopers: Abandoned.
I must admit though that I'm a little perplexed by your statement. I don't think it is appropriate to take into consideration the WC3 version. Marking it down because it feels like the original doesn't make sense to me. Most SC2 players have never tried WC3. The expression "if it ain't broke, don't fix it" comes to mind. In my humble opinion, it should solely be rated on its merits as a SC2 map.
Agree with the rest. I saw some of the Reviews and wanted to punch people in face for giving a 4/5 for some of the Terrain. -4/5 would be more like it for a lot of those. Just because the majority is bad, doesn't mean less bad is better in any respect.
Besides, rating systems are flawed in their very nature to begin with and it's probably better to give a First Impression like TotalBiscuit does.
TotalBiscuit does first impression videos because he has no time or willingness to make full reviews.
Also, rating systems are only flawed when they are rigid. A good rating system will change the criteria based on what it is reviewing. By very definition, the quality of something is determined by how well it does what you want it to do. Therefore, the criteria should be the aspects of the game/movie/whatever that you feel were important for the movie. You wouldn't have a "comedy" criterion for a horror film, but for a Sacha Baron Cohen film, you certainly would rate the movie on its comedic value. The criteria should also have only a suggestive influence on the final score for the work that is being reviewed, as the overall enjoyment should ultimately be more important than the other criteria.
I must admit though that I'm a little perplexed by your statement. I don't think it is appropriate to take into consideration the WC3 version. Marking it down because it feels like the original doesn't make sense to me. Most SC2 players have never tried WC3. The expression "if it ain't broke, don't fix it" comes to mind. In my humble opinion, it should solely be rated on its merits as a SC2 map.
I agree with this. I haven't played the SC2 edit of his map, I still play most of my custom maps on WC3. On WC3 it is a very unique TD/TW combo. With less variety on SC2, I would assume that would make it stick out even more above the competition. I can understand adding a comment to a rating saying "It has already been done, by the same guy" but it should not change the game rating. Game ratings are intended to tell potential players the best and worst parts of a game. Players aren't going to care if the editor has his work cut out for him because he was using an idea he already had.
I guess everyone reviews in their own way though. Some people only care about eye candy, and a 1/5 on visuals is worse than a 1/5 on gameplay. Other people are very happy with 5/5 gameplay, with a simple terrain, no imports, basic spells; but replayability.
Game ratings are intended to tell potential players the best and worst parts of a game. Players aren't going to care if the editor has his work cut out for him because he was using an idea he already had.
Well said.
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
Custom Maps
Diplomacy
DOTA/Hero
Tower Defense
Tug of War
Upcoming
The rating breakdowns can be found here.
Where did we get these ratings?
If you would like to volunteer for existing or future reviews, please let me know. [email protected]
Im not totally sure if the reviewers are doing this properly. Having equal importance on all those weird gategories makes this already so wrong.
And then you dont even think why the map is good or not, its more of a personal preference. Like in mineralz someone said the terrain is really plain...What would you expect. Its randomized and that adds a huge amount of replay value to the game. The game is simple and lacking some doodads really makes no difference in gameplay.
I suppose I just got a bit pissed since you didnt have proper statements to back up your scores. I personally couldnt even review sotis since I just played so much dota before that the sotis wont just do the trick anymore...
also whats up with the wierd review videos? watched 3 and they all had really different styles.
Just found out about this review stuff now :O
Cool idea but here's my thoughts (similar to zenx1 I guess):
Hard to judge maps of such different genre (under one number rating), right? Like me personally, I've made it well known that I can't stand turn-based text games. As good as Mafia is technically (and it's creator did a great job) I haven't enjoyed it and wouldn't enjoy it but that's just because I don't play those games... for the same reason I would probably not play Debates. Gotta be twitch-based!
So to me a 4.06 could be misleading? I don't know, I guess it's subjective.
Of course ratings are always going to be subjective, and I'm sure the rating system might have some flaws that will be worked out.
Regardless, I think having someone rating these maps in any capacity is really good for the community. Before SC2 was released, I thought the publishing system would also allow players to rate or review a map that's in the Blizzard database. That doesn't exist, and popularity is just a completely different metric from a real community opinion.
Thanks for doing this, please keep it up!
@SkrowFunk: Go I agree. Blizzard shown that HoTS will have a rating system, but anyway I'd like to see a more detailed system than simply giving the map 0 to 5 stars.
@Malpheus: Go Hey, who's in charge of picking maps and rating them? You guys should avoid the already popular ones and rate some unpopular maps (like mine) ;-)
@SoulFilcher: Go I think they keep a good balance of popular and unpopular maps. This way people can compare popular maps with unpopular maps.
I would be curious to see how Element TD fares.
Agree with the rest. I saw some of the Reviews and wanted to punch people in face for giving a 4/5 for some of the Terrain. -4/5 would be more like it for a lot of those. Just because the majority is bad, doesn't mean less bad is better in any respect.
Besides, rating systems are flawed in their very nature to begin with and it's probably better to give a First Impression like TotalBiscuit does.
Both magecraft and minerals have unfair disadvantages. Terrain. Neither of those maps can possibly have eye candy terrain. But the randomly generated arenas add alot of replayability. I think terrain should be replaced by appearance. Which would involve all eye candy material. Such as interface, effect and terrain. This would give games like magecraft and minerals a better balance in the rating.
@zenx1: Go
We did try to substantiate the ratings, alot of content was lost (articles mainly) due to a recent host change and software change.
As far as videos are concerned, the majority of the videos were simply intended to give a gameplay overview, and the majority were not our videos, normally another reviewer or the mapper's video. I am still working on consistent video presentation.
@OneTwoSC: Go
Over the weekend or near to it, I will totally change the format to address the different genre's of maps. When we initially conceptualized the process, it was meant to reward creativity and fun, hence the high mafia rating. We have yet to weight the six categories or change them, input is certainly valued in that respect.
@SkrowFunk: Go Ty for your support. We geared the reviews to address the map from the players perspective. Each of our reviews has at least one map maker and two players. I spent alot of time looking at discussions on bnet and here to determine this. The subjectivity of any review is absolutely unavoidable, essentially we are trying to figure out if the darn thing is fun. Instead of technically qualifying a rating, we use subjective statements from the reviewers to obtain a score. I would LOVE feedback on how to improve the accuracy of the five available statements within the six criteria.
@Karawasa: Go
It was started and halted till after your most recent big update, which has occurred. I'll put it after the two in que now. The most important thing I hope everyone grasps is we prioritize all requests. When there are no active requests, we simply troll to find the mapper making the most noise about a new map. If there isn't a prominent fuss being made over the latest, greatest, then we default to looking at the popularity list and up and coming list for interesting candidates.
@Stealthsam: Go
I think splitting the maps into genre's will help with this, but ultimately there are six factors. The appearance of a map is not limited to terrain. If you were to look at "trifaction" terrain you would be unimpressed from the editor view, but if you look at it third person, it looks pretty. It really is very subjective, ultimately that is why we use three people at a minimum to offset extremely subjective standpoints.
And all that being said, the most important aspect of the process is the challenge. If it's not accurate by your determination, please say so. Specifically. Each map reviewed gets a challenge if someone feels they got a bad bounce. The best example of this would be WW2 diplo, we rated it unfairly, it was pointed out, the rating was removed and the new one is pending my third rater finishing his end up.
ALSO I WOULD ABSOLUTELY LOVE VOLUNTEERS FOR THIS PROCESS, please let me know :)
Makes sense, thank you for doing that. I should note there is another huge update coming in the next day or two. I'm trying out TEAMS!
@Karawasa: Go
One of my reviewers loves ETD on WC and SC, I can't forget! We gonna put it on the list right now.
Index updated to reflect ww2 diplo in the ratings.
@Malpheus: Go
Updated to categories. Two more due this week. Elemental TD afterwards :)
With how many people are you testing the maps?
@MaNtEc0r: Go
Minimum of three. We're still doing it, holidays slowed us down a bit, I expect the Element TD review to be done this weekend.
@Malpheus: Go
Updated to include the Element TD Rating. The hard part to rate on this one was the fact it's really a sequel of the WC3 map in the truest form. Unlike a remake like Squad where someone did a legacy import of the terrain and built from there, emulating anothers work, this time we have the original maker remaking his own map on SC2... hard indeed.
It faired much better than the legion td clone, but I still felt like i was playing Element TD on WC3 ultimately.
Up Next Fleet Assault and Starship Troopers: Abandoned.
@Malpheus: Go
Thank you for reviewing it!
I must admit though that I'm a little perplexed by your statement. I don't think it is appropriate to take into consideration the WC3 version. Marking it down because it feels like the original doesn't make sense to me. Most SC2 players have never tried WC3. The expression "if it ain't broke, don't fix it" comes to mind. In my humble opinion, it should solely be rated on its merits as a SC2 map.
TotalBiscuit does first impression videos because he has no time or willingness to make full reviews.
Also, rating systems are only flawed when they are rigid. A good rating system will change the criteria based on what it is reviewing. By very definition, the quality of something is determined by how well it does what you want it to do. Therefore, the criteria should be the aspects of the game/movie/whatever that you feel were important for the movie. You wouldn't have a "comedy" criterion for a horror film, but for a Sacha Baron Cohen film, you certainly would rate the movie on its comedic value. The criteria should also have only a suggestive influence on the final score for the work that is being reviewed, as the overall enjoyment should ultimately be more important than the other criteria.
I agree with this. I haven't played the SC2 edit of his map, I still play most of my custom maps on WC3. On WC3 it is a very unique TD/TW combo. With less variety on SC2, I would assume that would make it stick out even more above the competition. I can understand adding a comment to a rating saying "It has already been done, by the same guy" but it should not change the game rating. Game ratings are intended to tell potential players the best and worst parts of a game. Players aren't going to care if the editor has his work cut out for him because he was using an idea he already had.
I guess everyone reviews in their own way though. Some people only care about eye candy, and a 1/5 on visuals is worse than a 1/5 on gameplay. Other people are very happy with 5/5 gameplay, with a simple terrain, no imports, basic spells; but replayability.
/end thought
Skype: [email protected] Current Project: Custom Hero Arena! US: battlenet:://starcraft/map/1/263274 EU: battlenet:://starcraft/map/2/186418
Well said.