Top 20 Maps Arrangement

[Shadow Fury]

It's been a while I keep observing the top 20 maps and I keep wondering "Are these maps really the best 20 ones in the whole Hive?". The answer is no. I mean, in the top 20 maps there aren't even all the director's cut maps, how come that? My own map reached the top 20 in less than a week or so because it is determined by the wrong factor: monthly downloads. Why should a map with the most recent downloads be considered the best? Director's cut and 5/5 maps are without any doubt the best maps in the whole hive but somehow they are not listed. I think that the top 20 maps shouldn't only be determined by the recent downloads but also by the mod's rating and the total rating from users for greater accuracy. Mod's rating takes 50%, user ratings 40% and downloads 10% (because as I implied above it's not really a determining factor in my opinion).

What do you think about this?

 

[WhiteFang]

I agree with you that the most downloads isn't really a determining factor and that it should be determined by ratings

 

[Wasabee]

That would be a good criteria because the Top 20 maps isn't just recent monthly downloads. But it doesn't mean that the number of downloads is not a determining factor considering the fact that a game that is very well made and polished can still be a crappy game if no one plays it.

 

[Ash]

Once I worked in a store and we had a metric called "best sellers". These would be the things that made the most money, calculated on a weekly basis, and were always skewed by the larger priced items like leather jackets and raw denim. You could sell one, maybe two of these in a week and they would amount to much more money than some of the other things we sold.

We had these t-shirts that would be regularly sold out due to their popularity, but they weren't the best sellers. Why? Because selling 20, 30 or even 40 of these things in a week didn't amount to the cost of a couple of leather jackets or raw jeans. I always thought this was weird. Just because we apply more value to something doesn't mean that they are always better. I think monthly downloads accounts for significantly more credit than you're giving it.

 

[Chaosy]

It's been a while I keep observing the top 20 maps and I keep wondering "Are these maps really the best 20 ones in the whole Hive?". The answer is no. I mean, in the top 20 maps there aren't even all the director's cut maps, how come that? My own map reached the top 20 in less than a week or so because it is determined by the wrong factor: monthly downloads. Why should a map with the most recent downloads be considered the best? Director's cut and 5/5 maps are without any doubt the best maps in the whole hive but somehow they are not listed. I think that the top 20 maps shouldn't only be determined by the recent downloads but also by the mod's rating and the total rating from users for greater accuracy. Mod's rating takes 50%, user ratings 40% and downloads 10% (because as I implied above it's not really a determining factor in my opinion).

What do you think about this?

What do you know, yet another topic where I disagree with you.

Firstly, yes I think it's stupid to judge my monthly downloads. That much I can agree on.

However judging blindly by rating is also wrong. "WTF are you saying Chaosy?!" you might think.
When you upload a map to the hive, you're basically trying to sell it to others (for no price but you get me.).
Naturally you do not have monopoly so there are other maps that people might download instead.
So in reality whoever sells the most copies is undoubtedly the winner. In actual products you have to calculate in the price, but since all maps are free we don't need to do that.

 

[Shar Dundred]

I cannot believe I'm saying this, but: I agree with Chaosy. Now I'll be having nightmares for the rest of my life...

I wouldn't call monthly downloads a "wrong factor". Monthly downloads doesn't mean the most recent downloads but rather number of downloads divided by number of months being online (= density of downloads).
The name "Top 20" might be quite misleading, but I think that the density is the best way to go. Ratings are individual and sometimes not trust-worthy. There are both high and low ratings at some resources by people without any reason, since they just want to give ratings.
Also, the density arrangement allows people who aren't using/having a Hive account to influence the rating as well.

 

[Shadow Fury]

Based on what you're saying, you're telling me that a DC map is worse than a map in the top 20 if it is not listed in there. As Chaosy said, maps are for free, products cost money. Clicking a download button doesn't take any effort and even if the map doesn't satisfy you, you delete it and it's over. However, in real life, you pay for the product. If you don't like it, it's not guaranteed you'll receive your money back. Based on that, people will buy what has been bought by the majority to be sure the money was spent on something worthy. However, with downloads, the same logic doesn't exactly apply.

Ratings may be inaccurate but so is the download density. You cannot tell me that a map that just got approved climbs the top 20 because it is a new one therefore everyone downloads it (that's what's currently happening). I can't buy that, really. Look at the second map in the top 20 (well, it's a campaign). It got a mod rating of 3/5 and an overall rating of 4.20/5.00 from users. Do you want to tell me that this campaign is better than, for example, Gaias Retaliation (mod rating 6/5 and user average rating 4.94/5.00)? This is just ridiculous...

Finally, I didn't want to ignore the downloads part so I proposed to let it account for 10% of the *score* to enter in the top 20.

 

[Ralle]

I think it would either be monthly downloads or rating. The reason why we don't use total downloads is that the longer the map has been here, the more downloads it has. A new map must have the same ability to get downloads even though it's new.

 

[Ash]

I really think my example is quite fitting, please don't overlook it...

 

[Shadow Fury]

I really think my example is quite fitting, please don't overlook it...

Your example is based on real life and money; it doesn't fit at all with my argument. Here products (virtual games) are for free.

 

[Shar Dundred]

Ash's example is still fitting and he has a point, SF, ehether you agree with it or not.

 

[Chaosy]

@SF
While in theory it could be true that an unworthy map could be in the top 20 by simply having a good concept and a wonderful thread layout but is horribly executed. And players wont notice until they play.

However I am yet to see such a scenario, all maps in the top 20 are really good maps. (at least when I looked the last time)

I also want to highlight that quality is not the same as fun.

Compare Minecraft with Crysis (1). You can buy both games for pretty much the same price, yet Crysis get 100x better graphics and cost fortune to develop. Meanwhile Minecraft was made in a Notch's (the creator) cellar.
Yet Minecraft likely got more copies sold. Quality does not equal fun, thus downloads mean more than rating.

 

[Ash]

YOU ACTIVATED MY TRAP CARD!

It's not about the money though, it's all about the idea that we attribute value to things that aren't necessarily reflective of how the rest of the universe views it. I mean, think about it. Citizen Kane is the undisputed most fantastic film ever created, but to your average cinemagoer they'll probably think it's trash. Do you catch my drift?

 

[Shadow Fury]

I do catch your point and understand what you mean. However, what I am talking about is that the top 20 isn't arranged in the right way based on fun and that stuff. I am sure that there are some maps that are worthy being in the top 20 because they are nicely done and very entertaining but the download density didn't allow that.

Compare Minecraft with Crysis (1). You can buy both games for pretty much the same price, yet Crysis get 100x better graphics and cost fortune to develop. Meanwhile Minecraft was made in a Notch's (the creator) cellar.
Yet Minecraft likely got more copies sold. Quality does not equal fun, thus downloads mean more than rating.

This is one type of unusual example but mostly this is not what happens (and yet again you are speaking about prices and money). The rating MEANS quality most of the times, otherwise you may come and tell me that 5/5 and 6/5 could be less enjoyable than 4/5 maps, a point with which I'd disagree. If a map scores so high there must be a reason. The game must be nearly flawless and very entertaining to achieve that rank. In addition, remember that money is also a very influencing factor so giving instances of products/games which cost money doesn't sound valid to me. As I stated before, clicking on the download button doesn't require any effort and you don't lose anything by doing so. Just as you admitted, unworthy maps may end up in the top 20 because they have good visuals but a cheap gameplay. Do you really want this to continue happening? The current top 20 maps keep gaining downloads because they are in the top 20 but if they weren't, I'd bet what you want that they wouldn't have such a high download density as they have now.

 

[Chaosy]

(and yet again you are speaking about prices and money)

I noted that the price is the same because all wc3 maps are the same as well since they are all free. It was to make the comparison even more logical.

The rating MEANS quality most of the times, otherwise you may come and tell me that 5/5 and 6/5 could be less enjoyable than 4/5 maps, a point with which I'd disagree. If a map scores so high there must be a reason. The game must be nearly flawless and very entertaining to achieve that rank.

I am no expert but I imagine you review a map like this:
Terrain, Gameplay, Story, Uniqueness

Now, say that a map get the following score.
Terrain 5/5
Gameplay 1/5
Story 3/5
Uniqueness 5/5
5 + 1 + 3 + 5 = 14
14 / 4 = 3.5 (4)

And then we got map B with the following scores:
Terrain 1/5
Gameplay 5/5
Story 1/5
Uniqueness 1/5
1 + 5 + 1 +1 = 8
8 / 4 = 2

In many cases I would prefer map B over map A, thus making it the more downloaded one despite being a bad map.
Examples of popular map that only got a good gameplay aspect and sucks at everything else would be the tom and jerry maps. They were constantly hosted at garena despite the overall bad quality apart from the fun gameplay.
This is why rating alone is -not- good.

 

[Shadow Fury]

I noted that the price is the same because all wc3 maps are the same as well since they are all free. It was to make the comparison even more logical.

"Free" is different from "price". If it doesn't cost anything, you don't lose anything by taking it. However, buying something lets you lose money. In that case, poor people will not buy something costy while with maps anyone can download them, increasing the number of downloads tremendously.

I am no expert but I imagine you review a map like this:
Terrain, Gameplay, Story, Uniqueness

Wrong. I noticed from my experience as a map reviewer and on online Warcraft gaming that nobody gives any interest to the story as long as the gameplay is good. TDs, for example, may have no story/recycled story behind them but would still be entertaining to play anyway. Uniqueness is also a point that people don't deem as important (I do however!) as many similar maps (AoS, TDs, Hero Arenas etc...) are played frequently even though the ideas could be recycled or unoriginal. Everything spins around gameplay and a little around terrain too.

Now, say that a map get the following score.
Terrain 5/5
Gameplay 1/5
Story 3/5
Uniqueness 5/5
5 + 1 + 3 + 5 = 14
14 / 4 = 3.5 (4)

And then we got map B with the following scores:
Terrain 1/5
Gameplay 5/5
Story 1/5
Uniqueness 1/5
1 + 5 + 1 +1 = 8
8 / 4 = 2

Assuming that both Map A and Map B have no bugs (because this is the third main category on which I base off my moderation), Map B would score the same as Map A. Remove story and uniqueness from both and we get 6/10 = 3/5. Since the gameplay of B is better, it will get more downloads so that "10%" I mentioned before would make the difference when the rating is the same for both (even if I cannot conceive a map with extraordinary terrain and very cheap gameplay).

In many cases I would prefer map B over map A, thus making it the more downloaded one despite being a bad map.
Examples of popular map that only got a good gameplay aspect and sucks at everything else would be the tom and jerry maps. They were constantly hosted at garena despite the overall bad quality apart from the fun gameplay.
This is why rating alone is -not- good.

Explained above.
Rating alone is -not- good, I agree, because sometimes they are inaccurate by users or the mod can be mistaken, that's why a 10% (we may change this number) will include download density. The 3 parts (mod rating, user average rating and download density) are subject to inaccuracies in all cases but summing them all means that if one is inaccurate, we hope that the other two would be accurate.

 

[Ardenian]

I don't think it is possible to create a Top 20 map list without disadvantage one aspect.

I think EVERYTHING should influence a maps position, though, despite the fact it is not possible to not disadvantage anything.

We would have to create a super-complicated mathematical function to get a map's 'value' as nearly as possible, to not be inaccurate. This would include a function which considers the download rate per month, the total download count and as well the time passed since a map has been published. And this is only one little point of the calculation.
There is mod rating ( you could even split it into sections like terrain, gameplay, features and so on),
then user rating which should also be tracked over time and count.

Then you would have to take care of the type and genre of the map, considering the number of players and compare the map to similar ones of similar types and genres.

I don't know, I don't think the Top 20 is viewed and used often.
It would be nice to have an appropriate one though, since it is kind of strange.

 

[StoPCampinGn00b]

A bit late reply but I want to insert my two cents. I promise that they're worth taking.

Reviewer rating rubric

I am no expert but I imagine you review a map like this:
Terrain, Gameplay, Story, Uniqueness

Now, say that a map get the following score.
Terrain 5/5
Gameplay 1/5
Story 3/5
Uniqueness 5/5
5 + 1 + 3 + 5 = 14
14 / 4 = 3.5 (4)

And then we got map B with the following scores:
Terrain 1/5
Gameplay 5/5
Story 1/5
Uniqueness 1/5
1 + 5 + 1 +1 = 8
8 / 4 = 2

In many cases I would prefer map B over map A, thus making it the more downloaded one despite being a bad map.
Examples of popular map that only got a good gameplay aspect and sucks at everything else would be the tom and jerry maps. They were constantly hosted at garena despite the overall bad quality apart from the fun gameplay.
This is why rating alone is -not- good.
__________________

I'm pretty sure no official reviewer nor moderator rates like this.

From what I see, most value it this way:
40-65% Gameplay
15-30% Uniqueness (Which can be eliminated by taking uniqueness into account for visuals & gameplay separately in a review)
15-35% Visuals

In truth, there are only two main categories that some split up. Gameplay (balance, creativity, bugs, polish) and Visuals (creativity, eye stimulation).

Maybe some rate all categories out of five equally, but they don't actually add it up as their rating for each category since they don't weigh the same.

Now I'm split here on the topic.

This is one type of unusual example but mostly this is not what happens (and yet again you are speaking about prices and money). The rating MEANS quality most of the times, otherwise you may come and tell me that 5/5 and 6/5 could be less enjoyable than 4/5 maps, a point with which I'd disagree. If a map scores so high there must be a reason. The game must be nearly flawless and very entertaining to achieve that rank. In addition, remember that money is also a very influencing factor so giving instances of products/games which cost money doesn't sound valid to me. As I stated before, clicking on the download button doesn't require any effort and you don't lose anything by doing so. Just as you admitted, unworthy maps may end up in the top 20 because they have good visuals but a cheap gameplay. Do you really want this to continue happening? The current top 20 maps keep gaining downloads because they are in the top 20 but if they weren't, I'd bet what you want that they wouldn't have such a high download density as they have now.

That is not an unusual example, it happens all the time. DotA and Call of Duty are some headliners.

What it boils down to are if simple, unoriginal, addictive, familiar type maps should get the spotlight over complex, critically acclaimed, hard to get used to maps and vise versa. It really depends who we're marketing to and who we think deserves to be on the top 20. What determines quality around here? Now we're back to square one.

Quality is subjective. So technically, the maps in the top 20 are the ones the public think are the best... or is it that they we're just shared and advertised with gimick? Should the public percentage be the factor that determines this? No? Why not? Some reasons could be that they aren't the ones keeping the community alive, but can that be justified? Maybe a better reason could be that map makers who got a high Hive rating probably worked a lot harder and they deserved to be rewarded more.

There's just so many arguments that would be semi-valid on either side. I think splitting it would satisfy everyone:

Top 20 maps of the month (ratings - 50% mod 50% user):
Top 20 maps of the month (downloads):

 

[Aeroblyctos]

It's good that we have opinion from two different sources: from map reviewers and the public(well somewhat). We're not biased.

For example I don't agree with many map reviews. Even some "Director Cut" reviews are bullshit.

 

[Avahor]

Well, it's better to add a 3rd rank category instead of rebuild top 20 map arrangement. In fact actual rank works fine for some particular searchs (sort by subcategories for example) and If I understand right the essence of this topic is about determining real good maps. New rank could be named "Consolidated maps" or "Hive's traditional maps" and it could be structured with many of the criterias posted here.

High Quality
-Director's cut
-Top 20 maps
-Consolidated maps.

 

[SpasMaster]

Yeah, it's kinda weird that a map that got uploaded fairly recently gets to the Top 20 that fast, but let's face it: whether it will stay there or not is determined by how much the users liked that map. At first it's a new resource and attracts interest - therefore downloads spike up. If the map turns out to be average/lacks replayability and/or updates it will quickly go down in "ranking".

I've been keeping a close eye to the "Top 20 Maps" section for many years now and I've seen this trend repeat itself over and over. About 90% of the time a map shows up, remains there for 1 week - 1 month and then it goes back down again due to the lack of updates and/or quality.

So, the best thing to do (as already someone suggested) is to have some minimal requirements for a map to pop up in the Top 20 list. But once it does, let it follow the current system of "downloads = higher rank", since downloads = interest.

 

[deepstrasz]

Just change it to top 20 downloaded maps to avoid confusion.

There would have to be some people to form a jury (mostly people who have experience with making maps) in order to actually decide the top for each month or for said period, something that would take a lot of time. Still that would not avoid subjectivity a 100% and would still be a democratic approach.