Removing Review Ratings

[Ralle]

We've decided to remove the ratings moderators and reviewers give to resources. This means that, no matter the person rating a resource, their votes will count equally. There are many reasons behind this, and a few are drawn attention to below.

1. We cannot be consistent over time.

The Hive has had a lot of moderators since we first opened shop to where we are now. It is simply not possible that the staffers aboard at the moment are giving ratings consistent to those that were aboard nearly ten years ago. The current system is flawed in this respect, but by getting rid of staff ratings we leave the decision about a resources quality to the user.

2. The opinion of a moderator/reviewer should not matter more than the users' when the users are the ones primarily going to use the resource.

Moderators and reviewers have many opinions on how they like to see resources turn out. This is all great. Feedback is wonderful. However, this site is for people to submit resources so other users can benefit from them. Therefore, the user matters just as much as a moderator. By having a moderator or reviewer aboard only to review or reject an item, we are roping in the users to help motivate and shape the product of the author's work.

3. Motivation.

Many users strive for a good approval from the staff, and this is cool - but a resource lasts longer than the approval. By making ratings general instead of specific, we hope to see authors continue to be interested in developing their work on THW.

A staffer will still decide where the resource goes, whether to approve, delete, or put it in the low quality section. They still post a review, but ultimately the only thing that differs from a normal user is where the resource ends up.

 

[A Void]

Nice! This should have been here right from the start. Old review and rating system had flaws where user reviews didn't line correctly with moderator ones which resulted in biased ratings on the resource.

 

[deepstrasz]

We've decided to remove the ratings moderators and reviewers give to resources.

Good. It was confusing especially when one moderator approved a resource and might have or not rated it.

 

[Kyrbi0]

This is a brilliant idea, and I'm quite glad to see it happen. I might even start rating resources now.

This basically puts the power of model ratings into the hands of the most concerned party, while leaving with the Stewards of the Site the ultimate "keep/discard" decision; it always struck me as weird/confusing, the dichotomy between moderator & user reviews.

It also solves the long-standing issues of "moderator review discrepancies" and "moderator preference"; especially the latter, where approval often hinged upon whether the mod liked it (not whether it was structurally sound & fulfilled the requirements of the section).

~~~

Question: what was my question?

Second question: Might be slightly off-topic, but can I say again that the name "Substandard/Too Simple" is just shy of offensive, in my opinion... There's got to be a better way to differentiate between models than that. I mean, we could go for "1st Class" and "2nd Class", or "Grade A" and "Grade B" (like eggs). Or heck, reverse it and turn the "Substandard" section into just the plain old "Model" section, while calling the currently "Model" secoin the "Superior Model" section or "High-Quality Model" or "Top Quality Model" or "Choice Model" or "Prime Model" or "Fine Model" or any number of synonyms. Basically lauding those who do better than average rather than shaming those who do worse than average.

Heck, the "Average Model Section" even sounds better. Still not ideal.

 

[Chaosy]

And there goes my only motivation to create stuff for the spell section.

Getting high ratings from a mod actually meant something.

 

[Kyrbi0]

You can still garner increased satisfaction from a good moderator review more than a good user review. And now you'll be saved from a personally biased moderator giving you a bad review and/or refusing to approve your spell, thus sinking any chances of being seen by anyone.

 

[Chaosy]

Not really the same thing anymore, it's no longer special.

I do like the part about mods not being able to give me shitty rating because reasons, but at the expense of mods being useless? (from my motivation standpoint xd)
Meh.

Also, this change will make searching harder.
Because the user ratings are not accurate. They are throwing 5/5's at their friends just because. However, mod reviews could be trusted most of the time. If I filter icons by 5/5 there will be good stuff. Filter by user ratings and you get 3/5 worthy stuff.

 

[deepstrasz]

Second question: Might be slightly off-topic, but can I say again that the name "Substandard/Too Simple" is just shy of offensive, in my opinion... There's got to be a better way to differentiate between models than that. I mean, we could go for "1st Class" and "2nd Class", or "Grade A" and "Grade B" (like eggs). Or heck, reverse it and turn the "Substandard" section into just the plain old "Model" section, while calling the currently "Model" secoin the "Superior Model" section or "High-Quality Model" or "Top Quality Model" or "Choice Model" or "Prime Model" or "Fine Model" or any number of synonyms. Basically lauding those who do better than average rather than shaming those who do worse than average.

No. Please. It's exactly like using the anatomic word for a sexual organ or a slang word. It's the same thing. It's like calling a janitor a cleaning department agent or something. It's all stupidly psychological.
However, I agree it shouldn't have two names but one. Substandard sounds fine.

Getting high ratings from a mod actually meant something.

A/one mod is not enough. You'll need more to at least make it feel like a jury.

 

[Chaosy]

Depends on the section.
For spells it's more or less objective.
The mod only needs to be skilled enough in the field, it's very hard to give reasons to give a person a low rating because you hate that one.

Either you have leaks or you don't.
Either you have good configuration or you don't.

It's rarely inaccurate in my experience so far.

I can see it being more of an issue in other sections where I have less experience.

 

[Kyrbi0]

Not really the same thing anymore, it's no longer special.

Yeah, except it isn't any different. The "specialness" is entirely subjective and all in your head. So you determine whether it's special or not.

Example: I can get a 5/5 rating for a resource from anyone, and sure it feels good. But a 5/5 from a friend of mine, or from a modder whose opinion I respect/value, will intrinsically mean a lot more/be 'special', to me. Doesn't matter what anyone else says/does (or the fact that in terms of the site's resource review calculations, it's all the same), it's still special... To me.

I do like the part about mods not being able to give me shitty rating because reasons, but at the expense of mods being useless? (from my motivation standpoint xd)
Meh.

They aren't useless. They are exactly as useful as everyone else.

Also, this change will make searching harder.
Because the user ratings are not accurate. They are throwing 5/5's at their friends just because. However, mod reviews could be trusted most of the time. If I filter icons by 5/5 there will be good stuff. Filter by user ratings and you get 3/5 worthy stuff.

Hm. I've never really searched by rating. I agree that people will be people; tending to be subjective. Just like Polls.

No. Please. It's exactly like using the anatomic word for a sexual organ or a slang word. It's the same thing. It's like calling a janitor a cleaning department agent or something. It's all stupidly psychological.

So you aren't affected by that kinda stuff. Good for you.

I'm not talking about using a euphemism or tumblr-special-snowflake garbage. It's just that there are loads of models in the Substandard/Too Simple section that are truly not "substandard", nor "too simple". They *are* the standard, if anything. It's just kind of like a jerk-move, in my opinion.

Depends on the section.
...
I can see it being more of an issue in other sections where I have less experience.

Yeah, the artistic resource sections (skins, icons, models especially) is pretty much the exact opposite. Lots of subjectivity.

 

[WhiteFang]

Overall,I do agree with this move.

Second question: Might be slightly off-topic, but can I say again that the name "Substandard/Too Simple" is just shy of offensive, in my opinion... There's got to be a better way to differentiate between models than that. I mean, we could go for "1st Class" and "2nd Class", or "Grade A" and "Grade B" (like eggs). Or heck, reverse it and turn the "Substandard" section into just the plain old "Model" section, while calling the currently "Model" secoin the "Superior Model" section or "High-Quality Model" or "Top Quality Model" or "Choice Model" or "Prime Model" or "Fine Model" or any number of synonyms. Basically lauding those who do better than average rather than shaming those who do worse than average.

I see what you are saying,and understand.However,if we add a 'superior model section' we will still have the same issue,just different.Those who made 'regular' models will still look bad and will envy those who have 'superior' models.

Also, this change will make searching harder.
Because the user ratings are not accurate. They are throwing 5/5's at their friends just because. However, mod reviews could be trusted most of the time. If I filter icons by 5/5 there will be good stuff. Filter by user ratings and you get 3/5 worthy stuff.

QFT.

At the end of the day,regardless if it's mods or user ratings,there will always be bias

Another thing is inexperience.Correct me if I'm wrong,but on the new hive you don't even have to comment on a resource to rate.Any random person can come and deal out 1/5s and 5/5s.Also,a inexperienced user may come around and look at a,let's say icon, and he sees nothing wrong with it.He then simply gives it a 5/5.Just because in his inexperience he could not see anything wrong

 

[Kazeon]

I was thinking to add my ratings stats to my portfolio, but nevermind. It's no longer reliable.

 

[Ralle]

Chaosy, a moderator is just a user who has janitorial duties and happens to be skilled in his field. Getting positive feedback from a skilled person should be all that matters if feedback is your motivation.

 

[Chaosy]

And the president of USA is just a man. Yet I would be way more happy if I got a compliment from the president, rather than my sister.

So I don't think it's that weird that I don't value the feedback of everyone equally.

It also goes beyond just words.
If you win, say a football competition you get a medal. Physical proof of what you achieved.
This used to be the case with mod ratings, but with them gone - the feeling of accomplishment is gone.

 

[Kyrbi0]

I see what you are saying,and understand.However,if we add a 'superior model section' we will still have the same issue,just different.Those who made 'regular' models will still look bad and will envy those who have 'superior' models.

I disagree. Making a model that is graded as "regular" is not so painful as being graded "substandard/too simple". Literally "not up to the standards".

And even if that's the case, that's why I first suggested "Class" or "Grade". Grade A/B is fine. First/Second Class is fine.

And the president of USA is just a man. Yet I would be way more happy if I got a compliment from the president, rather than my sister.

So I don't think it's that weird that I don't value the feedback of everyone equally.

Precisely. And that intrinsic value you ascribe to certain people cannot be taken away, regardless of what the site does. So you're good there.

 

[Chaosy]

Read the the edited part of the post.
It's not the same.

 

[Deleted member 219079]

I don't like this "blending in with the herd" thing. Moderator should be high authority and carry the responsibilities of his own field. Rather step down as a mod than blame the system (you said "we've decided" so mods have discussed this).

Users should be expected to be immature, giving ratings of 5/5, 4/5 or 1/5, or nothing at all, without further evaluation. Also prone to bias, e.g. model by -Grendel- gets them hard before even seeing whether the asset is functional.

I'd cut down from the other end: dump down user ratings (e.g. like/dislike) and make the rating spectrum wider for mods, or amplify it in another way.

 

[Chaosy]

I'd cut down from the other end: dump down user ratings (e.g. like/dislike) and make the rating spectrum wider for mods, or amplify it in another way.

Aye. Too biased.

 

[SpasMaster]

So, ummm. I'll share my opinion on this topic.
It will be primarily focused on Map Moderation, since well... that's what I do.

Overall I can see reasoning behind this change. A moderator might decide that a map is not "good", while the community loves it. A good example could be very popular maps like DotA, Elemental TD, Castle Fight, Enfo's Team Survival, etc - those maps would possibly get average ratings here, whilst adored by the community.

However, I personally always saw Moderator Ratings as a rating given to a map based on the THW requirements/standards. I remember few years back seeing a topic where DotA was discussed (the wc3 mod) and someone said that it probably wouldn't get more than 4/5 on Hive. Which made sense, if the Moderator Rating symbolised what I pesonally saw it as - the rating of the map based on THW criteria. DotA lacks in several fields that are highly valued in the Hiveworkshop modding community. Does that make it a bad map? Hell, no. Yet, the Moderator Rating would have judged the map based on "modding" criteria. Not "popularity". A map like Gaias or TKoK, or even that 'Diablo 3 into Warcraft 3' mod have a much greater modding value in them compared to DotA - which lacks in those fields compared to the quoted maps.
There was a nice distinction between User Rating which symbolised the rating of the players and the Moderator Rating which symbolised the "modding" rating.
At least, that's how I've always seen it.

All that ^ is my opining on the topic from a general point of view.

Now, let me get all personal. My personal issues shouldn't impact how the entire moderation system functions, yet I'd like to share them.

Rambling

I joined in 2010 when getting your map approved took months and you had to have invested at least some time in your resource for it to be approved. Maps were put on hold and rejected all over the place, due to simple terrains, code leaks, these sort of things. Even map descriptions! Many people were not familiar with the site rules and requirements. Guess what, though. After waiting for months, a moderator came, slapped you in the face by telling you that you need to improve on various fields: terrain, coding, description, custom creatures/spells/heroes. Then he told you that he would pay you a visit soonTM. From my experience and point of view this was incredibly motivational. There was a 'THW judge' who knew and imposed the THW rules and requirements and you had to work on your resource for it to be approved. Oh, yes - ratings didn't come until much later. But I guess WC3 is a bit less lively nowadays and all of that sounds too harsh - I get it. However, as someone who has followed all those rules strictly and as someone who has worked on insane amount of features for years, I feel somewhat bad regarding this change. Let me explain: I have recieved a Director's Cut Rating for Sunken City in 2013 after @-Kobas- gave me big lists of things I must work on and improve on to get the rating. And I did work hard. And I did clear every single note from his list. And I got the rating. And now it's gone.

I can see why.. I can see the reasons. Yet I feel like I've lost something I worked hard for to get. Don't get me wrong, I don't want a freaking medal or something. It's just that I can't shake the feeling of losing something.

 

[Shar Dundred]

Meh.

 

[Kazeon]

Haha.. My maps too! All those sparkling stars, got overshadowed by 3/5 user ratings >.> (Kidding inside) But I also kinda feel like lost something. But really, it's not a big deal. As long as it's an improvement, I will get along

 

[deepstrasz]

It's just that there are loads of models in the Substandard/Too Simple section that are truly not "substandard", nor "too simple". They *are* the standard, if anything. It's just kind of like a jerk-move, in my opinion.

That's why I think a jury of moderators+/-reviewers is needed to determine the final outcome of a resource's classification. One moderator might just think it's substandard or standard. How's that for objectivity?

if we add a 'superior model section' we will still have the same issue,just different.Those who made 'regular' models will still look bad and will envy those who have 'superior' models.

What's the Director's Cut absence into this have to do?

Correct me if I'm wrong,but on the new hive you don't even have to comment on a resource to rate

Yes it's anonymous. Not even if a comment is left does the creator of that resource know what rating the said user chose (remember the thumbs?). I usually write the rating myself in a comment. I think that's the honourable way to go with it.

Getting positive feedback from a skilled person should be all that matters if feedback is your motivation.

Yes, though they are the lesser percentage of the whole that rates. I don't really care that much about that specification. What I do care about is for people to have some dignity to show their rating and reasons for it in my face.

And the president of USA is just a man. Yet I would be way more happy if I got a compliment from the president, rather than my sister.

As with kings. It is a problem of psychology. It all depends on that person. It's not the authority that matters it's the person. The president can be a bad or uneducated person or on the contrary. You get my drift.

I don't like this "blending in with the herd" thing. Moderator should be high authority and carry the responsibilities of his own field. Rather step down as a mod than blame the system (you said "we've decided" so mods have discussed this).

What, you don't like democracy now?

Hell, no. Yet, the Moderator Rating would have judged the map based on "modding" criteria. Not "popularity". A map like Gaias or TKoK, or even that 'Diablo 3 into Warcraft 3' mod have a much greater modding value in them compared to DotA - which lacks in those fields compared to the quoted maps.

I disagree comparing two different types of map genres to each other. DotA can get a 5/5 for it's originality (in case it's not a ripoff of something earlier to it) of the gameplay, multiple heroes and whatnot. If you just want to compare the hours of work for maps, then that is another story and should not interfere with the rating. Why couldn't a Warcraft tetris map get a 5/5 and Sunken City also 5/5? It's like saying a carrot is worse than a pear because you have to cut its tail and peel it before eating it.

From my experience and point of view this was incredibly motivational.

More like lack of moderators and their activity.

Yet I feel like I've lost something I worked hard for to get.

Sorry, but a Director's Cut is more than one can hope for.

Meh.

He knows what he's talking about.

 

[Deleted member 219079]

I'm more for the representative democracy, if even that.

Of course it's easy to appeal when promoting individualism. I prefer ppl with a pair though.

 

[Ralle]

To brighten the deal a bit, I will share something on my TODO list.

We plan on making a way to mark posts as reviews. Posts marked as review will be referenced in the top below the bundle with a summary, rating, submitter and date (kind of like moderator review and rating). This will help those looking for resources to make the right decision while also possibly be a pad on the back for the author(s) of the bundle. This is also why the moderator review was converted to a post.

This means that moderators/reviewers 1. decide what is approved/rejected and 2. what is a thorough review and what isn't. I believe by then we should have a way to request that certain posts get marked as review for your bundles. It does not mean that moderators/reviewers are forced to write such reviews for every bundle they see.

 

[deepstrasz]

It does not mean that moderators/reviewers are forced to write such reviews for every bundle they see.

Well, that cuts them some slack, doesn't it? I mean why should they even bother to test the map themselves if underprivileged user posts can be marked as reviews? The idea might work for me if a reviewer/moderator proves that the map was tested by a privileged user without copying the text from a normal user or using it as the main review/way of accepting/rejecting the respective resource.

 

[Ralle]

Imagine this.
I test a map for five minutes. I look at the reviews and find one I think fits my thoughts and the rating seems fair. I mark it as a review.

This does not mean the staff shouldn't test things themselves. It just allows diversity in opinion to display multiple opinions about a thing.

 

[deepstrasz]

Imagine this.

To be clear. I'm against moderators/reviewers not having their own written opinion about the resource besides the marked ones.

I test a map for five minutes.

It's what I'm afraid of.

 

[Kyrbi0]

That's why I think a jury of moderators+/-reviewers is needed to determine the final outcome of a resource's classification. One moderator might just think it's substandard or standard. How's that for objectivity?

A jury would be nice, but I'm not sure it's not asking too much. It's hard enough to get one moderator to finish the massive backlog.

The "subjectivity/objectivity" issue is essentially immaterial (up to a point); a review is subjective if it has an emotional basis, doesn't have a 'scale'/'numbers'. It's objective if they avoid preferences/biases/predispositions (& often, go off of a rubric or something).

So if the moderator has & uses a "rubric" (0-5 for mesh, 0-5 for animations, etc), the review will likely be more objective. If they don't & instead review like "well this model looked cool but i don't like forest trollz so unapproved", that's more subjective.

What's the Director's Cut absence into this have to do?

Wait, did they say they were doing away with the Director's Cut?

Yes it's anonymous. Not even if a comment is left does the creator of that resource know what rating the said user chose (remember the thumbs?). I usually write the rating myself in a comment. I think that's the honourable way to go with it.

You have precisely highlighted one of the primary issues of Polls as well as reviews.

Sorry, but a Director's Cut is more than one can hope for.

How so?

I don't like this "blending in with the herd" thing. Moderator should be high authority and carry the responsibilities of his own field. Rather step down as a mod than blame the system (you said "we've decided" so mods have discussed this).

I think you expect too much from moderators. Not just the "they are people too" and not even "it's just a hobby". But like Ralle said, many moderators have come & gone, and it's difficult (if impossible) to enforce certain expectations upon them.

 

[Shar Dundred]

I test a map for five minutes. I look at the reviews and find one I think fits my thoughts and the rating seems fair. I mark it as a review.

That is bullshit, you cannot judge what review is "the one most fitting" after five minutes.


I don't really have an opinion about this entire matter, I honestly couldn't care less, but this
is just ridiculous.

Either remove reviews entirely or not at all.
Don't replace the old system with something like that.

 

[Deleted member 219079]

I think you expect too much from moderators. Not just the "they are people too" and not even "it's just a hobby". But like Ralle said, many moderators have come & gone, and it's difficult (if impossible) to enforce certain expectations upon them.

I think our picture of a user differs a little?

Hidden

It's either mods giving as proper reviews as they can or a broken system. Yes mod reviews differ imminently over time, but users' differ the moment they see the author.

 

[Deleted member 238226]

i think matters like this should be discussed with the user before an action will is taken.. ><

hope this is for the best!

edit : grammar mistake lol..

 

[deepstrasz]

Wait, did they say they were doing away with the Director's Cut?

No. I meant the Director's Cut is for those maps rated the best. No need for too many/other classes/grades.

How so?

Well, what more can you expect from a moderation perspective?

i think matters like this should be discussed with the user before an action will is taken.. >

What's that supposed to mean? Negotiation?

 

[Murlocologist]

User ratings are very little reliable especially when they are usually given by criteria that got nothing to do with actual quality of resource. And rating resources is something very few people do in the first place, most resources got one or two ratings. If this was website where every resource got at least hundred ratings than you could actually search for high quality resources by ratings given. Also moderators and reviews are left with task that requires little skill, either put resource in approved section or substandard.

 

[The_Silent]

I realized a reason I'm not too happy with this.
I doesn't allow a lot of room for improvement on the side of the content creator once their resource is approved. Since this effectively reduces the rating system to a binary Approved/Not Approved, there is no reason left to work on a model once it is approved. Which either means we have to be a lot harder on content creator, and approve less resource, or simply ignore it, and accept some resource is never going to get better.

Secondly, I actually used Moderator Ratings as a filter option ALOT in the past.

 

[Chaosy]

Also moderators and reviews are left with task that requires little skill, either put resource in approved section or substandard.

Quoted for truth.
Pretty much anyone can do this in any section. You don't need to be very good at drawing to see the difference between a great skin and a shitty one.

@The_Silent
Very true, if you update a model (For example) very few will come back and change the ratings.

And me too, it's the only filter I used.

 

[IcemanBo]

@The_Silent the points have some truth, but are the same I think. Because in case you become motivated to update your resource for a better reviewer rating, you actually also only want that your resource gains some public fame and will be listed at "top resources". So basicly for the search/filters, and not for quality. But if you focus on quality, you will like give a damn (extremly said) on the pure ratings, but will instead rely on critiques and reviews.

Each user has probably a goal somewhere in between the both extremas, they want some good status in the resource list (fame), but also provide some decent and useful resurce (quality).

With removing the reviewer rating now, we have less power to (seemingly) highlight our resources to be outstanding of the rest, but must rely on the overall community rating which seems not special, but just common.

In my opinion we need to focus more on quality in our reesource sections than on some mental ego pushing-- so there, I weight reviews much more than plain ratings, even they come from reviewers.

For Maps, though, I think it makes sense to also rely on ratings at filter. Because the user often doesn't search for a specific resource (like in spells/icons/... usually), but sometimes just want to play famous maps that seem to be best of the bests. And filter by ratings from offical reviewers gives even some more confidence to the users.
But... Ralle made a very good point in the first post about why we should actually not rely extremly on reviewer ratings (just like on any other ratings):

1. We cannot be consistent over time.

... and this is very true. So now the community has equal powers in ratings, just like the staff has, and we should learn to use this possiblity as hive member, and also more rely on this method as a resource user. I think it's something good.

 

[Murlocologist]

@The_Silent the points have some truth, but are the same I think. Because in case you become motivated to update your resource for a better reviewer rating, you actually also only want that your resource gains some public fame and will be listed at "top resources". So basicly for the search/filters, and not for quality. But if you focus on quality, you will like give a damn (extremly said) on the pure ratings, but will instead rely on critiques and reviews.

Each user has probably a goal somewhere in between the both extremas, they want some good status in the resource list (fame), but also provide some decent and useful resurce (quality).

With removing the reviewer rating now, we have less power to (seemingly) highlight our resources to be outstanding of the rest, but must rely on the overall community rating which seems not special, but just common.

In my opinion we need to focus more on quality in our reesource sections than on some mental ego pushing-- so there, I weight reviews much more than plain ratings, even they come from reviewers.

For Maps, though, I think it makes sense to also rely on ratings at filter. Because the user often doesn't search for a specific resource (like in spells/icons/... usually), but sometimes just want to play famous maps that seem to be best of the bests. And filter by ratings from offical reviewers gives even some more confidence to the users.
But... Ralle made a very good point in the first post about why we should actually not rely extremly on reviewer ratings:
... and this is very true. So now the community has equal powers in ratings, just like the staff has, and we should learn to use this possiblity as hive member, and also more rely on this method as a resource user. I think it's something good.

Motivation for one to improve resource is irrelevant. Quality is what matters in the end and if uploader improved it only so it can be rated 5, not 3, no one really cares. Again, what motivates one to upload resource is irrelevant and I don't see how, or why, would you ever want to manipulate with the aspect of personal motivation. And reason why ratings exist is primary to filter trash from good stuff, at least from my stance.

On side note, I always though there was too much traffic in resources section, not that I'm for some sort of elitism when judging the resource, but too many crappy resources being put with really good ones in same box is the case, so in the end we get 300 pages of mix between shits no one should ever use for any project and good stuff people forgot or overseen as they are stuck on 250th page.

 

[IcemanBo]

Sorry, I'm not sure to what point your post is. Or you just enhanced the opinion?

 

[Murlocologist]

I'm referring to your post in which your starting position is that we need to focus more on quality in our resource sections than on some mental ego pushing.

 

[deepstrasz]

I'm referring to your post in which your starting position is that we need to focus more on quality in our resource sections than on some mental ego pushing.

Isn't what moderators and reviewers (those with the specified position, not normal users) are doing, quality control? It's not like the users decide if a resource gets approved or not.

 

[Murlocologist]

Isn't what moderators and reviewers (those with the specified position, not normal users) are doing, quality control? It's not like the users decide if a resource gets approved or not.

Did you even read my previous post?

 

[deepstrasz]

Did you even read my previous post?

I was taking a general remark. And motivation is only irrelevant when it comes to rating, yes but it's really useful. I've seen many maps become better after moderators/reviewers put them in the Awaiting Update section while giving them the necessary feedback with the things to improve.

On side note, I always though there was too much traffic in resources section, not that I'm for some sort of elitism when judging the resource, but too many crappy resources being put with really good ones in same box is the case, so in the end we get 300 pages of mix between shits no one should ever use for any project and good stuff people forgot or overseen as they are stuck on 250th page.

The problem is that a lot of the stuff that was approved way back in the day, needs reassessment which of course, I don't think any moderator/reviewer would want to be a part of.

 

[Murlocologist]

And motivation is only irrelevant when it comes to rating, yes but it's really useful.

I don't get it.

 

[deepstrasz]

I don't get it.

Most people will rate the resource the first time they see it and not (again) after it is updated. That majority basically puts the specified resource up or down on the list of "quality". However, there's a higher chance moderators and reviewers will actually raise their previous rating of a resource after it has been updated.

 

[Chaosy]

Isn't what moderators and reviewers (those with the specified position, not normal users) are doing, quality control?

No, that's what they were doing.

his means that moderators/reviewers 1. decide what is approved/rejected

As someone else (can't be bothered to go back and check xd) said, anyone can do this job now.

 

[Murlocologist]

Most people will rate the resource the first time they see it and not (again) after it is updated. That majority basically puts the specified resource up or down on the list of "quality". However, there's a higher chance moderators and reviewers will actually raise their previous rating of a resource after it has been updated.

This got nothing to do with anything I said lol.

What difference would it make if mod would change grade since their vote is equal as any random user's?

 

[deepstrasz]

No, that's what they were doing.

I was speaking as if the change hadn't been implemented yet.

As someone else (can't be bothered to go back and check xd) said, anyone can do this job now.

I doubt moderators will want to take their time moving ancient maps in the Substandard section or out of it judging by 0.5% people who actually play and then review them.
Which means, the only way to pacify this is to just only base things by user rating and not moderator/reviewer's because the strict system came later than when a ton of early maps were uploaded.

What difference would it make if mod would change grade since their vote is equal as any random user's?

I meant that moderators shouldn't lose their authority of pushing creators to update their stuff for the good of the Hive quality even if most users don't mind specifics in their reviews.

 

[Murlocologist]

I meant that moderators shouldn't lose their authority of pushing creators to update their stuff for the good of the Hive quality even if most users don't mind specifics in their reviews.

What? Your stance is confusing to me and it is quite of a challenge to understand what your overall point is supposed to be.

 

[deepstrasz]

What? Your stance is confusing to me and it is quite of a challenge to understand what your overall point is supposed to be.

Well here, from the beginning. You can read previous posts you know but hey.

We've decided to remove the ratings moderators and reviewers give to resources.

This is a "good" thing because the number of moderators actually rating a resource was 95% one.

1. We cannot be consistent over time.

Especially when there are maps that didn't have the chance to get the moderation of the system the Hive has had for some years. Doubtful that anyone even unprivileged users will want to rerate/rereview those.

but by getting rid of staff ratings we leave the decision about a resources quality to the user.

This is not really pertinent since a majority of users do not objectify quality.

It is simply not possible that the staffers aboard at the moment are giving ratings consistent to those that were aboard nearly ten years ago.

Mostly because back then a resource's quality was differently judged. The most that have to suffer from this are the maps. Back then a high rated RPG was made with little to no external resources than what the World Editor offered. Even the spells, abilities and other gameplay mechanics were not changed from that of the original game but that sort of map still got a high rating judging by the time's standards.

Moderators and reviewers have many opinions on how they like to see resources turn out. This is all great. Feedback is wonderful. However, this site is for people to submit resources so other users can benefit from them.

That sounds like there should be no quality control however.... (look below)

Therefore, the user matters just as much as a moderator. By having a moderator or reviewer aboard only to review or reject an item, we are roping in the users to help motivate and shape the product of the author's work.

Which is a bit confusing since moderators/reviewers still decide the outcome of a resource's placement/acceptance?

Many users strive for a good approval from the staff, and this is cool - but a resource lasts longer than the approval. By making ratings general instead of specific, we hope to see authors continue to be interested in developing their work on THW.

To be frank, I don't really get this one. Are user ratings different in the Pending/Awaiting Update section than in the Approved or Substandard section?
Anyways, I see it the way that a (more than one to be desired) moderator/reviewer should be the first person to push the creator of a resource to polish it since users mostly don't care about details that much.

A staffer will still decide where the resource goes, whether to approve, delete, or put it in the low quality section. They still post a review, but ultimately the only thing that differs from a normal user is where the resource ends up.

Now, honestly, at first I thought this thread was about just removing the moderator rating that

We plan on making a way to mark posts as reviews. Posts marked as review will be referenced in the top below the bundle with a summary, rating, submitter and date (kind of like moderator review and rating). This will help those looking for resources to make the right decision while also possibly be a pad on the back for the author(s) of the bundle. This is also why the moderator review was converted to a post.

This means that moderators/reviewers 1. decide what is approved/rejected and 2. what is a thorough review and what isn't. I believe by then we should have a way to request that certain posts get marked as review for your bundles. It does not mean that moderators/reviewers are forced to write such reviews for every bundle they see.

Which is neat and all but cuts too much slack for the mods/revs. I however would agree with this for the old resources that have no proper moderation. There user critiques would ease the pain of time wasting reviewing again even if that would be the honourable thing to do.

So, my standpoints are:
-a jury of moderators for the outcome of a resource (which is almost impossible to do; gathering the proper number of people)
-mods/revs should still have their own resource critique alongside those belonging to normal users
-the above mentioned authority should still have an important role in pushing creators to polish their work

Do I need to make it clearer?

Why are most moderators and reviewers seemingly on abstention?

 

[IcemanBo]

I'm not sure you understand the difference between ratings and reviews--maybe also read my previous post again.

Reviews/feedback is for make statements about the real quality.
Ratings is to see how much the folks, the entire user base, likes your product.

And reviewers/moderators still ensure quality and have expertise in the respective section. They just don't rate it seperatly anymore.