Solutions to Map Reviewing Issue

[Captain Griffen]

1) THW's system looks retarded if not all maps are reviewed, due to the rediculous build up.

2) The number of maps to be reviewed is massive, requiring a large number of mods.

3) The number of high enough quality map reviewers is not that big.

4) Getting in low quality reviewers defeats the point, is not effective at useful feedback (may even give worse than no feedback - bad feedback), and massively reduces confidence in ratings.

5) Good feedback normally means you have to tell people their map is shit.

I'm not going to provide a solution. Why? Because I don't want to lead THW, I don't want to set policy on THW, and I damned well don't want to turn THW in WC3C. Diversity is a good thing.

I put it forward, however, that reviewing ever map properly, or even with anything approaching properly, is not possible.

Please think before posting, and consider that I am possibly the most experienced map reviewer in WC3 modding - I know how to review, and I know how to give the feedback people *need* to hear, rather than what they want to hear, and that is what I'm doing it. You may not like it, but we need to deal with reality.

 

[brad.dude03]

Ralle has been talking of outsourcing map moderation to the crowd. I tend to agree with him.

 

[Linaze]

The problem to some extent lies in the enormous amount of uploaded maps, whereof 90% are no where near acceptable, and yet need to be tested, therefore I suppose the map moderators can easily get tired of what they're doing. That, plus some moderators don't enjoy to test every genre, making it even worse.

 

[ragingspeedhorn]

Wc3sear.ch's no-testing-needed worked fine. If a map is crap it'll end up reported anyway and then get removed since. For a site that accepts mostly anything I don't see the need for testing before approval, just test the stuff that people report as crap.

 

[Linaze]

Wc3sear.ch's no-testing-needed worked fine. If a map is crap it'll end up reported anyway and then get removed since. For a site that accepts mostly anything I don't see the need for testing before approval, just test the stuff that people report as crap.

Unfortunately it's true, despite almost how bad a map is, the map moderators of THW test it, write two sentences and approve it. Accept all should work almost the same.

 

[Ash]

im slightly drunk at the moment, so i'll sav e a serious reply for later.

imo map reviewing here is little to none existant. points arent reviews, and the cohorts of idiots arent mod-worthy.

it needs [another] rejingle, im afraid.

 

[Hakeem]

To be honest, I'm not entirely sure what a map mod does besides make sure the map has the proper permission to be uploaded, write a review, and give it a rating.

• If a map is discovered to be here without proper permission, then it can be removed.
• Even with moderators, it is infeasible for every map to get a proper review. Not saying we should stop reviewing all maps, because when a really good map comes along, a review and Director's cut would be nice.
• I never cared for ratings because I often don't agree. I'm not saying we should remove the rating system, or the moderator rating, because there are (I assume) people who "care" about that. User votes wont be far behind an unrated map.

So yeah, what they said.

 

[Rui]

What is the point of this topic? We know the situation and all these philosophies.

The Map Moderators spend a lot of time playing the Maps and planning the way they're going to say things.
Maps here don't 'need' a review. I don't mind if they pile up again. I prefer that over low-quality reviews.

(...) points arent reviews (...)

Points or written texts are the same. They both tell you what to improve and what's fine. This is most likely an attempt to turn an empty thread into a discussion of what is and what is not a review. If that is so, I will not reply. We've gone over this already -- you have your review style, I have mine. Period.

 

[PurplePoot]

Back to the point of the thread, I think that a wc3sear.ch style system is becoming more and more feasible.

 

[Ash]

you have your review style, I have mine. Period.

Calling your 'review' a 'style' just made me nearly fall off my chair lolling. The best point of this thread so far is as follows:

Back to the point of the thread, I think that a wc3sear.ch style system is becoming more and more feasible.

 

[Linaze]

Well both me an raging said it before. As it looks now, the wc3sear.ch system is soon the only possible outcome.

 

[Ash]

They only reason it is the only possible outcome is because mods don't take the time to actually review maps.

You get crap things like this and that.

If we adopt a wc3sear.ch styled outlook on maps, then when mods actually do review things I don't want to see any of the crap we currently have.

 

[Asomath]

They only reason it is the only possible outcome is because mods don't take the time to actually review maps.

You get crap things like this and that.

If we adopt a wc3sear.ch styled outlook on maps, then when mods actually do review things I don't want to see any of the crap we currently have.

Wait, are you calling my map crap or the moderation. Cause I put lots of time into that, and to create a detailed story in that. Now the moderation itself wasnt helpful, so I would give you credit for a good judgement.

~Asomath

P.S. Both you and Septimus give thorough reviews most of the time, which includes both good and bad reccomendations to the map. So, I didnt see a problem in the second example.

 

[Ash]

Wait, are you calling my map crap or the moderation. Cause I put lots of time into that, and to create a detailed story in that. Now the moderation itself wasnt helpful, so I would give you credit for a good judgement.

~Asomath

P.S. Both you and Septimus give thorough reviews most of the time, which includes both good and bad reccomendations to the map. So, I didnt see a problem in the second example.

The critique, if you want to call it that.

 

[Airandius]

They only reason it is the only possible outcome is because mods don't take the time to actually review maps.

You get crap things like this and that.

If we adopt a wc3sear.ch styled outlook on maps, then when mods actually do review things I don't want to see any of the crap we currently have.

You have to consider that it's VERY hard and that it takes ALOT of time to moderate a map properly, Also I have given some solutions to reduce strain for moderators to increase the quality on the hive.
You can find those in the other tread called; Mod Suggestion

We have come with a good idea (in most peoples few) to remove the delete option and make people upload only finished maps, not alpha's or beta's that need testing.

~ Airandius

 

[FamousPker49]

You have to consider that it's VERY hard and that it takes ALOT of time to moderate a map properly, Also I have given some solutions to reduce strain for moderators to increase the quality on the hive.
You can find those in the other tread called; Mod Suggestion

We have come with a good idea (in most peoples few) to remove the delete option and make people upload only finished maps, not alpha's or beta's that need testing.

~ Airandius

I think Ash of all people would know this, as he has been a map mod twice now, and resigned. He always gives a very detailed and good map review, whether or not the review is positive is dependent on the quality of the map.

But I do not think it is good to have the moderators review each and every map, because I can easily understand the eventually tiring of low quality maps that are submitted.

I am not exactly sure how Wc3sear.ch worked, but it seems like the prevailing idea here.

 

[PurplePoot]

wc3search had no approval process, but had moderators to go back and sift through everything which looked like it needed sifting through.

It was pretty nice most of the time, but annoying sometimes... I once spent a day cleaning out broken linked maps from the older recesses of it. Not the funnest day of my life.

 

[Rising_Dusk]

I think Ash of all people would know this, as he has been a map mod twice now, and resigned. He always gives a very detailed and good map review, whether or not the review is positive is dependent on the quality of the map.

How does the fact that he's a great reviewer but an unreliable moderator matter at all? It's obvious to even the half-witted that reviewing takes time and effort, specifically for it to be meaningful. Regardless, though, due to the sheer influx of maps, it is impossible for mods to keep up -- Particularly when no one with the capacity to review well wants to be on the staff in the first place.

Just let this take it's course and we'll see where the hive stands after that. The obvious solution is to remove the easily accessible ability for a user to delete their maps and force them to go through a staff member. (Am I repeating myself or something?) If that's done, then the staff knows before the fact what people are trying to abuse the feature. This is why there is an admin dedicated specifically to the community, God forbid he actually has to do something.

 

[Mecheon]

The obvious solution is to remove the easily accessible ability for a user to delete their maps and force them to go through a staff member.

Yeah, well I oppose this decision. I will oppose this tooth and nail. This is removing a vital right of people's uploading capabilities and so help me, if they actually put this on the website I am outta here

We'd be no better than bloody Xgm if we pulled a stupid move like that

 

[Grim]

I agree with Ralle and Brad Dude about passing map moderation to the crowd.

The issues I see with that is:
- Biast Opinins for or against.
- Lack of motivation.

Everyone should be allowed to test and review each map. Moderators however should be able to scan through the posts and take into account there general trend in opinion of the more respected users (eg. users with above 50 repuation). At the bottom of each post, the volunteer should simply say whether the map should be passed or not at that point. If it is clear the general trend in opinion is for or against it then the moderator will simply finish the job quickly.
To reward users, for each map they choose to test, maybe add a new volunteer reputation system (like Vrep ). For this, only moderators can give this type of reputation, and it is given only if you helped test the map, review it to a certain extent, and possibly come back and test a few updates.
I dunno if any of you guys would understand this system.
Vrep would probably be a bit more respectable then normal rep, mainly because normal rep can just be thrown about for any reason. (btw... vrep wouldnt replace normal rep and I dont think Vrep should show next to users posts... just in their profile so no one gets too confused ^^)

At the end of the day... this could simply be another shit idea ^^

EDIT:
Might be harsh... but instead of the above... maybe neg rep for a completley SHIT map that wastes mods time?
Because people might upload maps that they havent even tested themselves... also, maybe after a mpa is rejected a 2 day ban on posting anymore? Might be harsh... but could easily convince users to test there work first... and give them 2 days to improve there map rather then spam it back up...

 

[The Last Istari]

Might be harsh... but instead of the above... maybe neg rep for a completley SHIT map that wastes mods time?
Because people might upload maps that they havent even tested themselves... also, maybe after a mpa is rejected a 2 day ban on posting anymore? Might be harsh... but could easily convince users to test there work first... and give them 2 days to improve there map rather then spam it back up...

No, that can't be done. That not only can completely demoralize people, but the definition of a "SHIT map" is mostly different with everyone.

As such, if a map reviewer reviews said map, it's decently made (in my opinion) bue because of the reviewer's constant success and knowledge of mapmaking, his standards go higher and he considers that map a "SHIT map" in which neg rep is dispensed.

So yeah, it could cause standards to skyrocket, which may sound like a good idea, but it could keep people who actually are here for good reasons to think this place is horribly elitist and leave.

 

[Steel_Stallion]

A "SHIT" map should be rejected immediately, but we should then have a global definition of what a "SHIT" map is.

What ever happened to that rule where you can't steal maps? What the fuck is Life of a peasant Megaman X?

Its attack of the fucking clones, that's what it is. THOSE maps deserve to be rejected on sight, because they are pretty obvious from the name and mini map preview.

 

[Septimus]

A "SHIT" map should be rejected immediately, but we should then have a global definition of what a "SHIT" map is.

What ever happened to that rule where you can't steal maps? What the fuck is Life of a peasant Megaman X?

Its attack of the fucking clones, that's what it is. THOSE maps deserve to be rejected on sight, because they are pretty obvious from the name and mini map preview.

The problem is, it is very hard to determine whether the map is created by the person or been stolen.

Some stolen map have gone through at least 90% changes, making it hard to be acknowledge whether it is genuine or stolen.

While some LOAP map is really created by the person, is just the name of the map similiar.

 

[VGsatomi]

wc3search had no approval process, but had moderators to go back and sift through everything which looked like it needed sifting through.

It was pretty nice most of the time, but annoying sometimes... I once spent a day cleaning out broken linked maps from the older recesses of it. Not the funnest day of my life.

Tell me about it...I went though ALL 5000+ OF THEM making sure every link worked and removing that ones that were broken....took me several days.

back to topic...

Given the volume and rapid map intake rate of this site, a system that gives users a chance to report rule-breaking maps rather then mods searching though every last one of them would certainly be the most logical choice in my opinion. It did work very well at wc3search, and I think it would be just as effective here. As it stand snow the mods are way too overburdened.
-VGsatomi

 

[Ralle]

The way you report posts on the boards does also work in maps, and you can report the map itself with the same button.

I don't think it would be too hard to moderate all the pending maps. I am considering to moderate a page every day. I would write a review, a SHORT one. Rather a few comments on why I acted like I did. If somebody wanna review a resource, they can write a post.

 

[Steel_Stallion]

The problem is, it is very hard to determine whether the map is created by the person or been stolen.

Some stolen map have gone through at least 90% changes, making it hard to be acknowledge whether it is genuine or stolen.

While some LOAP map is really created by the person, is just the name of the map similiar.

Actually, most are very easy to determine. Rejection could also be based on the maps poor quality because it is exactly the same as the others with some imported models and a flame strike that does 999999999 dmg.

FFS

 

[Grim]

The way you report posts on the boards does also work in maps, and you can report the map itself with the same button.

I don't think it would be too hard to moderate all the pending maps. I am considering to moderate a page every day. I would write a review, a SHORT one. Rather a few comments on why I acted like I did. If somebody wanna review a resource, they can write a post.

Uuuh, I don't mean to argue with you ralle, but I think there are, like 22 pages of pending maps backlog? That's gotta be pretty tough to catch up with?
Didn't someone earlier on in this thread mention tempoary mods? Maybe ask 4/5 or so well respected users whether they would be willing to take the title of a temp?
This would basically mean, they've agreed to spend lots of their time testing the maps accepting/rejecting them and once the backlog has been shrunk to a certain extent remove the map mod power?

 

[Septimus]

Actually, most are very easy to determine. Rejection could also be based on the maps poor quality because it is exactly the same as the others with some imported models and a flame strike that does 999999999 dmg.

FFS

Low quality map is quite easy to determine, but stolen map isn't.

 

[Ash]

Low quality map is quite easy to determine, but stolen map isn't.

Just thought I'd jump in and say a few words to this.

LoaP maps are quite clearly recognized as 'stolen' -- if you'd want to call it that, I prefer badly edited, myself -- because they have exactly the same piece of terrain around the centre, or bottom, usually, of the map.

However, that doesn't mean that the map is edited without consent. For all we know, the original LoaP map could, very well, be open source.

 

[Septimus]

Just thought I'd jump in and say a few words to this.

LoaP maps are quite clearly recognized as 'stolen' -- if you'd want to call it that, I prefer badly edited, myself -- because they have exactly the same piece of terrain around the centre, or bottom, usually, of the map.

However, that doesn't mean that the map is edited without consent. For all we know, the original LoaP map could, very well, be open source.

That is why it is hard to determine it cause of this reason.

1) Could be open sources
2) The original author might have give permission to 1 person only.
3) The original author cannot be contact to determine permission have been granted to the user or not.

Well, not all LOAP is stolen. Sometimes I compare their trigger and some have a completely opposite triggering style.

 

[Steel_Stallion]

Low quality map is quite easy to determine, but stolen map isn't.

"Low quality" is opinion based.

Anything original, no matter the quality, if it is working as it should be and is up to minimum standards than it can be approved. But these unoriginal garbage Loaps, Vampirisim, DBZ and etc we keep getting by the dozens I want to see rejected.

 

[The Last Istari]

Oh, and don't forget the Naruto maps, those always turn out awful.

 

[Grim]

How about this then...
New Map Uploading Rules should be drawn up.
1) Map Terrain should be made from SCRATCH - Unless:
I) You have screenshotted evidence that you have permission to use the terrain
II) The true and original creator of the terrain is unknown and the general community would consider the map to be open-source.

2) The Map must be completley finished... If not it should be posted in the map development section on the forums.

3) A Full Map description should be available so normal users see what they're downloading (This might be a rule already.. but there are still maps with no description :/)

These are just my opinions... I think these would help to improve the odd map but something else would be much better...

 

[Ash]

The map rules aren't at fault, we'd drafted them up to a tea. What we need to address, though, is the way they're moderated.

Do we set the status to 'approved' automatically, but give it a distinction if it passes a moderator review?

Do we remove the delete button because 'people re-upload their map' (without putting too much personal spin onto it, I'm strongly, strongly against this)?

Do we include the users more in Moderation? An example I used a while back is this:

5 User reviews = Approved
2 Minimod reviews = Approved
1 Mod review = Approved

And obviously this can be used in conjunction with point number one, with a tiny bit of tweaking.

 

[Grim]

5 User reviews = Approved
2 Minimod reviews = Approved
1 Mod review = Approved

I agree with this
However, many maps won't get 5 User reviews :/

 

[Andriejj]

And who'd be keeping record of those user reviews, searching through the comment spam? ;P

 

[Ash]

Remove the option for a comment a map saying 'LOL DIS IS GD I PLY IT ON BNET WIF MAYTEZ'. /Shrug.

 

[ragingspeedhorn]

Why? The authors might like getting comments saying their map is good without it nescesarily being a throrough review. It is a comment system after all.

 

[Steel_Stallion]

Moderation on the comments given on a map is way too much work, even if it is mostly spam. The moderators have their hands full as it is I doubt they have time to moderate comments on a map

 

[Ash]

In that case, give the users a system like the minimods have.

 

[Grim]

It's not that difficult to see what is spam and what is a review...

OR, why not just have a little option available next to the map rating where they have to select one of 3 options:
- Requires Approval
- Requires Improving
- Requires Rejecting

The statistics of the vote would show what options had been selected and the mod can simply decide which option would be best to take.

(By the way Ash - I like ya blog )

 

[Ash]

In that case, give the users a system like the minimods have.

It's not that difficult to see what is spam and what is a review...

OR, why not just have a little option available next to the map rating where they have to select one of 3 options:
- Requires Approval
- Requires Improving
- Requires Rejecting

The statistics of the vote would show what options had been selected and the mod can simply decide which option would be best to take.

One in the same thing, really. But I think if we're providing them with an option of that -- as opposed to a separate box -- then we should at least add into the rules saying 'you can only vote for approval/improving/rejecting if you provide a review with your comment'.

 

[Newuser]

One in the same thing, really. But I think if we're providing them with an option of that -- as opposed to a separate box -- then we should at least add into the rules saying 'you can only vote for approval/improving/rejecting if you provide a review with your comment'.

How about add a button named "Write a review", which allows users to write a "formal" review and rate the map.

Reviews and comments would be separated. Reviews can only be written once(but can be updated) per user. Reviews need to actually contain a review, if it doesn't, it could have a "report" button for mod delete.

Anyway, the review system would make it very easy to organize the review.

It'd have a "Pros:", "Cons/Bugs:", "Main Review", and "Suggestions for Improvements:" "text block" which would make things easier for organization.

Users would still be able to rate the map from 1-5 if they simply post a comment(no review needed).

However, users who reviewed the map can also "rate if the map works" from a scale of 1-5, this is not the same thing as regular map ratings, it's just a rating to from 1-5 to see if the map works or not.

5 means the map is working as intended, no bugs, no flaws, etc.
3-4 means the map might have some bugs.
1-2 means the map has a lot of bugs and flaws and it needs to be fixed quickly.

If a map has at least 3 reviews, and the map has an average "map working" rating of at least 3(after 3 reviews), the map automatically gets approved BUT however, the map can get "deapproved" if the "map working rating" drops from 3.

Also, only users with 10 posts, and users who registered and joined the site for at least a month can write reviews for maps.

To recap:

1. There should be a "Write a review" button for maps, for users to write a review.
2. The review should make it easy for users to organize their reviews, it should have a "Pros", "Cons/Bugs", "Map Review", and "Suggestions for Improvements" textbox, allowing users to easily organize their reviews.
3. There are two ratings for each and every map, the "regular map rating", and the "if the map works rating".

"Regular Map Rating" - This means how much the user enjoyed the map or likes the map. Users can rate the map simply by posting a comment, no review needed.

"If the map works Rating" - This is a rating from 1-5 to check if the map is "working as intended".
5 means the map is working as intended, no bugs, no flaws, etc.
3-4 means the map might have some bugs.
1-2 means the map has a lot of bugs and flaws and it needs to be fixed quickly.

Only users who "formally reviewed" the map can rate in the "If the map works Rating".

4. If a map has at least 3 reviews, and at least a rating of 3+ on the "If the map works rating", the map automatically gets approved, however the map can get "disapproved" if the "If the map works rating" drops from 3.

 

[Septimus]

I do not think it is necessary, a user could have just send the link of their review to map moderator. If the information are accurate and honest, it would be approve. That is what I did when I was still a user.

 

[Ash]

How about add a button named "Write a review", which allows users to write a "formal" review and rate the map. ...

... If a map has at least 3 reviews, and at least a rating of 3+ on the "If the map works rating", the map automatically gets approved, however the map can get "disapproved" if the "If the map works rating" drops from 3.

That's exactly what I said, just a lot more long winded

Basically, user reviews should be in the region of 5ish people, simply due to the fact that -- and no, I'm not saying this is representative of the majority of the hive -- a canny few people are complete and utter fucktards.

If they write a review, it should appear in the same region as comments, but if the map is viewed by a moderator then he should be able to see a sign saying 'X user reviews for approval', same goes for rejection and the same goes for improve.

I'll go into more detail about the rejection and improve when we all agree that this is a good system, though.

I do not think it is necessary, a user could have just send the link of their review to map moderator. If the information are accurate and honest, it would be approve. That is what I did when I was still a user.

We can't do that because it'd be insanely hard. A moderator would receive at least 5 reviews for one map, whilst receiving a heck of a lot more for ever other map that needs approval.

 

[kewlthis]

They only reason it is the only possible outcome is because mods don't take the time to actually review maps.

You get crap things like this and that.

If we adopt a wc3sear.ch styled outlook on maps, then when mods actually do review things I don't want to see any of the crap we currently have.

I agree the should be somekind of requirements, when posting spells/maps. It should atleast give decent amount of information on the map. Dunno what else.
Maybe normal people should be allowed to rate maps too for +rep (atleast giving +rep for that would be more meaningfull than random repping)

 

[Ash]

Read the rest of the thread, kewlthis... D:

 

[Steel_Stallion]

Aren't you gone yet Ash? Why does everyone do some dramatic shit and come back anyway

I don't mean that in an offensive way btw


But um, wont people just spam reviews like they spam comments?

comment:

"nice map man omg u got my fav model u rock i host al time dud kk "

Review:

"nice map man omg u got my fav model I give u +5 rating i host al time dud kk

Rating: 5"

IMO, we should have a system similar to "Medivhs tower" where people who want to be able to review need to apply for it and be approved.

 

[Ash]

I'm on a break for a month, and I'm overseeing all the threads I went in before I sod off.

I could go sooner, if you'd like?

 

[Steel_Stallion]

I don't really mean it that way, I was just wondering since you said you were leaving