1. Updated Resource Submission Rules: All model & skin resource submissions must now include an in-game screenshot. This is to help speed up the moderation process and to show how the model and/or texture looks like from the in-game camera.
    Dismiss Notice
  2. DID YOU KNOW - That you can unlock new rank icons by posting on the forums or winning contests? Click here to customize your rank or read our User Rank Policy to see a list of ranks that you can unlock. Have you won a contest and still haven't received your rank award? Then please contact the administration.
    Dismiss Notice
  3. Lead your forces to battle in the 15th Techtree Contest. The call is yours, commander!
    Dismiss Notice
  4. The reforging of the races is complete. Come see the 14th Techtree Contest Results.
    Dismiss Notice
  5. It's time to choose your horse in the race - the 32nd Modeling Contest Poll is up!
    Dismiss Notice
  6. Check out the Staff job openings thread.
    Dismiss Notice
Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

Map Quality

Discussion in 'Site Discussion' started by Ralle, Feb 28, 2010.

?

How should we find quality?

Poll closed Mar 14, 2010.
  1. Filter by raw user ratings

    4 vote(s)
    9.1%
  2. Filter by the rating of the users whose posts are marked as reviews

    32 vote(s)
    72.7%
  3. Slow moderation with a longer and longer queue. It doesn't really work.

    8 vote(s)
    18.2%
  1. Teaspoon

    Teaspoon

    Joined:
    Oct 1, 2009
    Messages:
    510
    Resources:
    13
    Models:
    13
    Resources:
    13
    I like it...
     
  2. Linaze

    Linaze

    Joined:
    Feb 28, 2007
    Messages:
    2,991
    Resources:
    2
    Models:
    1
    Tutorials:
    1
    Resources:
    2
    I don't.
    Fun Factor > All
     
  3. Teaspoon

    Teaspoon

    Joined:
    Oct 1, 2009
    Messages:
    510
    Resources:
    13
    Models:
    13
    Resources:
    13
    Yeah the fun-factor should be a bigger part of the map, otherwise I like it.
     
  4. Linaze

    Linaze

    Joined:
    Feb 28, 2007
    Messages:
    2,991
    Resources:
    2
    Models:
    1
    Tutorials:
    1
    Resources:
    2
    I've seen no map mod use a "score-system", what does that say? They suck.
     
  5. Joe-black-5

    Joe-black-5

    Joined:
    Apr 27, 2008
    Messages:
    1,315
    Resources:
    1
    Maps:
    1
    Resources:
    1
    Moderation + Score-systems = fail in my opinion
     
  6. Pyritie

    Pyritie

    Joined:
    Nov 26, 2006
    Messages:
    11,356
    Resources:
    60
    Models:
    30
    Icons:
    9
    Packs:
    3
    Skins:
    12
    Tools:
    1
    Maps:
    1
    Tutorials:
    4
    Resources:
    60
    I really don't like score systems people do
     
  7. En_Fuego

    En_Fuego

    Joined:
    Sep 3, 2006
    Messages:
    1,186
    Resources:
    0
    Resources:
    0
    Score systems are okay. I just don't like how difficult they may be to balance.

    For example, some maps might have an overly-extensive use for heroes while another map may not have any heroes at all. Same thing goes for items.

    As for the fun factor, how can you determne a balance between how much the fun factor is worth and the rest of the points? I think Septimus used to use something like this when "grading" maps and it never really felt fair. I mean, you'd lose points for not implementing AI/single player compatibility...would this be the same?

    Also, Linaze, thanks for the quote.
     
  8. Rui

    Rui

    Joined:
    Jan 7, 2005
    Messages:
    8,506
    Resources:
    8
    Maps:
    8
    Resources:
    8
    Like I have said numerous times, the map reviewers should be chosen by the Map Moderators. There are users who write better comments than others. In my opinion, those should be privileged.

    Also, people, for the last time, we are not turning our Map Section into a wc3s system.
    The maps which lack descriptions are there because they were imported and we keep them in respect for wc3sear.ch's content.
    Our current map moderation system requires a certain (and inexistent) number of professional map reviewers to work and we are aware that it isn't working. We could either do something about it or do nothing about it and let the maps pile up (therefore, a wc3s system). Conscious that doing nothing and giving up usually achieves nothing, we chose the first option.
    Plus, if no one has yet noticed, we do have a Reviewed status now. Those maps are in agreement with all the rules and the majority of them have, on top of that, been reviewed by more than a single individual. This is not bureaucracy. It is, to answer Teaspoon's statement, giving certain maps the due respect they deserve. We are not highlighting DotA just because it the most played map on Battle.Net. The Kingdom of Kaliron, on the other hand, pushes Warcraft III to a level I have never seen before and is constantly using the engine to its limits; that is, to use Warcraft III at its full potential. I consider that more valuable than creating an unorganized object data made of masses of buildings and «races» whose balance relation is nonexistent, the type of object data you'll find in most Tower Defense maps. Do not think I'm distinguishing them because of the map type, because I am not. If I find a TD that has been, at least, properly balanced, I will be sure to give it its due rating.
     
  9. DragoonZombie

    DragoonZombie

    Joined:
    Mar 24, 2005
    Messages:
    436
    Resources:
    3
    Models:
    1
    Icons:
    1
    Tutorials:
    1
    Resources:
    3
    That is like completely different from what I think. I for one don't really like TKoK. I agree on you that it indeed uses the full potential of wc3, but does this really make this a good map? A quality map? Your definition of quality and mine differs, so IMO it is not good to let only specific users review a map.
    But what you can do is seperate comments and reviews. So that you have the option to either post a comment or a review. Then users could mark a review as either helpful or not helpful and depending on that it will be displayed on top or bottom of the review list. Reviews with lots of 'not helpful' clicks can then be checked by mods or as you want users with 'Map Reviewer' rank to see whether it is spam and if it is delete it. Or just make it automatically after a certain amount of 'not helpful' clicks.

    But there is one thing you have to differentiate. That is critics/improvements and reviews. I think you cannot review a beta/alpha status map. The moment you review a beta map and that is being updated to a release version your review will most likely not match anymore. So to prevent that you could implement a simple checkbox that says [Beta] and if it is checked (in upload process) it will disable the option to post reviews. Of course you will be able to check/uncheck the beta status when you decide to update/edit your map.
     
  10. syltman

    syltman

    Joined:
    Jun 13, 2007
    Messages:
    1,326
    Resources:
    0
    Resources:
    0
    Note: I didn't read most of the post, only those at the end.

    I think a map should be reviewed in quality, while a map being fun should certainly be a big bonus it is not what makes a map. For instance a map could be fun but impossible to learn due to lack of documentation. I think that making the quality maps receiving a better rating will encourage people to increase the quality of their maps and it generally improves our database as a whole since people wont publish bad stuff.

    Beta/alpha/incomplete maps should not be allowed at all, what we get then is a bunch of incomplete junk in our database.
    Adding to what I just said, I find it harsh that incomplete maps actually gets approved and reviewed faster then other maps, like for instance the Diablo 3 borderlands map.

    I think we should only allow complete maps with an above average quality to be approved. But I think that the current system could work if we stick to it this way.

    Then also perhaps minimods should be able to report stuff for having to little documentation/info, too bad quality, etc. so you got more specific stuff then just if this map deserves to be approved or not. That way it's easier for mods to see if a map deserves to be approved without having to try it.
     
  11. Riptokus

    Riptokus

    Joined:
    Aug 7, 2007
    Messages:
    112
    Resources:
    2
    Maps:
    1
    Tutorials:
    1
    Resources:
    2
    Here's how -I- would like reviews to count.

    I want to sort map reviews by more than it's score.
    A map with 10 votes that give it an average of 4.49 is 100% better than a map with 1 rating that gives it a 5.
    As such, I want to put parameters into my sorting. I want to be able to sort maps by rating, but with a download and number of ratings limited.
    It's something I like to have the power over too, not something that can get an arbitrary limit. I'd also occasionally like to sort that list by other parameters.
    Right now, all we can sort by is the categories, and not use any of the other figures which are supposed to help us choose to play the map.

    Next we go into reviews.

    I have found a serious problem in our maps and the quality of them. It all stems from one source. Noobs that post their maps in finished resources without any development time in Map Development. These people use finished resources to develop their maps. Some are blatiently flaunting the rules, calling their maps alpha maps. Others are pretending they are within the rules by calling unplayable maps beta maps.

    As such, I say remove this statement from completion, to remove the confusion:
    If it just stands at this:
    Beta maps that meet the standards of uploading fit into this category, so we haven't changed our high standard any higher/lower, but removed the justification for a lot of people arguing with me that their map meets the standard.

    Also, we need to add a statement in there like this:
    Reviews in the map section are for the benefit of people who are trying to decide if they want to download the map or not, and not for the map creator or development of the map. If you feel your map needs further development, please use the Map Development forum.

    Finally, let's place pending on the LAST PAGE of searches unless someone deliberately asks for pending maps. Pending maps are often the criminals in making our reputation look bad.

    Also, in uploading, make a question "Did this map have a map development page in the Hive Workshop forum" and ask for a link to the page. If it didn't have this, one recommendation for rejection, no matter whose, can instantly reject it. This mechanism would do wonders for preventing a lot of people using the maps section for developing the maps.

    There shouldn't be a pending purge, instead you should get a small team together who are willing to run a quality check on all the current maps, and they recommend the maps for approval/rejection. Their word would be taken as law for any rejections. This would reduce the pending to something manageable enough for our map moderators to examine them.

    Finally, an approved map should not be sacred. Things do get by screening all the time. A map should be "Report"-able, with a drop-down on exactly what the violation is, and a comment box to explain it in detail. Approved maps should always have the benefit of the doubt, but with that kind of system, it shouldn't take more than 10 minutes for a moderator to be able to identify if the report was valid, and probably less than 3 seconds ("Report: Presentation. Comment: The mapmaker is a jerk!")

    Of course a tracking system can be made for that to see people who falsely report or are wrong a lot. Also the map comments are a good indicator if it's someone who is just grinding an axe or something that is legitimate.

    We should also double the amount of map mods we have.
     
  12. dead-man-walking

    dead-man-walking

    Joined:
    Nov 30, 2009
    Messages:
    1,575
    Resources:
    0
    Resources:
    0
    Seriously? We need to Triple the amount. We only have one Full-Time Map Moderator right now.
     
  13. shiiK

    shiiK

    Joined:
    Sep 15, 2008
    Messages:
    1,745
    Resources:
    54
    Models:
    1
    Icons:
    30
    Skins:
    22
    Tutorials:
    1
    Resources:
    54
    Didn't read through all the posts, but out of the tree options, I think qualified reviews should determine the rating of which maps are sorted after. These reviews would have to be thorough and proper. 'twould probably be a good idea to promote people to be map reviewers (they wouldn't moderate maps, ie approve/reject etc, but would rate the already approved maps) whom would provide a detailed review of every aspect of the map for users to read before they download, a lot like how movies and games are reviewed.

    General user ratings are always messed up. Some people rate 5 just 'cause they can, others vote 1 because they can, and a select few actually tries to make a serious rating.
     
  14. Ralle

    Ralle

    Owner

    Joined:
    Oct 6, 2004
    Messages:
    11,372
    Resources:
    22
    Tools:
    3
    Maps:
    5
    Tutorials:
    14
    Resources:
    22
    OK I have decided that I want to continue on the plan i was having. I was unsure, you helped me. I am going with the 2nd option. I will implement it when I get better and have more time.

    Sorry to those of you I told that it would be done on a certain day two hours later. I became unsure and made this thread instead that day.
     
  15. Fanny.Shaver

    Fanny.Shaver

    Joined:
    Oct 4, 2009
    Messages:
    171
    Resources:
    0
    Resources:
    0
    Will you have enough people to mark map reviews as a reviews?

    I applied for map mod some time ago. I had enough determination to test maps etc, Ralle rejected me due to my name on this forum.
     
  16. Rui

    Rui

    Joined:
    Jan 7, 2005
    Messages:
    8,506
    Resources:
    8
    Maps:
    8
    Resources:
    8
    Like I said, the maps on Reviewed status have been reviewed by more than a single person.
    I'm trying not to go off-topic here, but I gave TKoK its rating because on top of using the engine to its full potential, it is fun to play and has a relatively decent storyline – on a few RPGs, the story is completely despicable. This is enough for me.
    To finalize, however, the majority of the people who think criteria differs are usually wrong. I'm not saying you are – you didn't mention what was yours –, but I think that, in the end, criteria tends to be quite similar.

    This is a good idea, but I do not think vBulletin has a system that allows something like this. I don't know if you were around at the time, but in the past we have used a resource section that was separate from vBulletin and, if I'm not wrong, Ralle did not appreciate of the lack of functionality between it and vB. In my opinion, it looks better now, even though choosing to use vB costed us the possible option to differentiate comments, which is what you're asking.

    When I review maps, I always choose those from the moderation queue. These maps are the oldest of the pending list. I always try to give the newest maps time to receive the first waves of critique before analyzing them myself.
    By the way, I always mention the version of the map on my reviews.


    This is a good idea. I hope Ralle implements it.

    I agree, I suppose, even though this is not a democracy; on such a system, the accused may argue that the law says this and this is susceptible of being misinterpreted and is therefore forgiven. But on a «friendly dictatorship», like the Hive, moderators can make their interpretation of the rules without being questioned; and, if by any chance someone contacts the administration, the moderator will simply explain it to them and the topic is closed.

    Agreed.

    I can't see why.
    Anyway, click on Approved Maps if you do not want maps with Pending status to be displayed.

    This would be useful. Good idea.

    The mini-moderator system would leave a comment on the moderation panel that the moderators could easily examine. Nevertheless, the maps do have a Report button that you can use. But I wouldn't make it reject anything automatically, much because of the type of people you mentioned on the second paragraph (of the text I'm quoting).

    In the past, I have suggested that ratings be given solely by map reviewers. But for the obvious lack of people to do this, it never went forth.
    Your idea is good, though, I'd like these reviews to be displayed after the map description. Lately, I've been trying to mention moderator reviews on the moderator comment field, inclusively.