Removing Review Ratings

[Kam]

This is a positive step.

I think as some have stated it will affect the rating score over time and lower the usefulness of the star rating.

 

[Chaosy]

Reviews/feedback is for make statements about the real quality.

Except anyone can write a review, so it's the same as the user ratings.
Completely useless.

 

[deepstrasz]

I'm not sure you understand the difference between ratings and reviews

I know the difference. What's your point?

And reviewers/moderators still ensure quality and have expertise in the respective section. They just don't rate it seperatly anymore.

Yes, that's the good thing but then:
Removing Review Ratings
and
Removing Review Ratings

I'm saying I agree with everything the chief wrote in the initial post which is basically the removing of the moderator 5 stars rating that's under the one for users to vote but I don't agree with moderators/reviewers marking normal user reviews above their own critique. That means, mods/revs should still do their "job" and not base most judgement on just any user reviews.

 

[Ralle]

but I don't agree with moderators/reviewers marking normal user reviews above their own critique. That means, mods/revs should still do their "job" and not base most judgement on just any user reviews.

Moderators have for a while quoted user reviews when they see them as fitting their own opinion, this will allow them to do that more directly. I probably shouldn't have said the "test a map for 5 minutes" part though.

 

[IcemanBo]

The method of approvement has not much to do with reviewer ratings.

It's very valid if a reviewer reads and trusts an other user's review and critiques as long it's serious. The mod has to read & evaluate it, and if he think it's okay, then it's okay if he takes it in consideration.
Of course it can be checked here and there, but other's people review surely can be as qualified as ones by staff. So if the approving moderator sees it alike, it is fine.

What Ralle meant is I believe, that the reviewer team will do easier and faster to evaluate such situations if reviews are bundled under the submission.

Except anyone can write a review, so it's the same as the user ratings.
Completely useless.

You know, you can speak bad about anything you want. If you think critiques and reviews are senseless, then the talk is senseless, too. Sure everyone can try to make a review, but who says that the appoving mod will see it as a valid review to addopt?

 

[Shar Dundred]

Welcome to Epicwar Reloaded, gentlemen.

 

[Chaosy]

Welcome to Epicwar Reloaded, gentlemen.

Don't even.

You know, you can speak bad about anything you want. If you think critiques and reviews are senseless, then the talk is senseless, too. Sure everyone can try to make a review, but who says that the appoving mod will see it as a valid review to addopt?

Let's say we have a submission with five reviews, the mod picks one. But the other four are complete garbage. (don't make me point fingers)
Which would make the review 'score' inaccurate which is now how it should be according to:

Reviews/feedback is for make statements about the real quality.

So it's not the same thing as the mod rating was, it's just like the user ratings.

 

[Aleister]

As A Void said, this should have been here right from the start.

Maps, models, skins, icons and all other resources, are ART, and as such, must be assessed independently by each member of this site.

Feeling motivated by a mod rating, it's like feeling motivated because the kindergarten teacher puts a "happy face seal" on your homework. Come on kids, stop crying.

 

[Chaosy]

I'm sure getting the nobel prize is not a big deal either because the ones selecting the winners are not the common folk, but a small selection of people.
(or pretty much any award, really)

Yes, this includes art.
For example the Oscars which included but is not limited to art is decided by vote of 5000-ish people (as of 2012, it may be more now)
Source: Academy Awards - Wikipedia, the free encyclopedia

 

[Shar Dundred]

Come on kids, stop crying.

How old are you again to think you have the right to call anyone "kid"?

 

[Ralle]

Let's say we have a submission with five reviews, the mod picks one. But the other four are complete garbage. (don't make me point fingers)

Chaosy, you're wrong. The moderator/reviewer will do as he/she has done so far. Write a review. Now it won't just be a post however. It will be shown in the top with the bundle. However, there is the ability to have multiple reviews. If two reviewers disagree, they can both write a review. Users can hope for their posts to be marked as reviews (by a reviewer/moderator) as aspiring reviewers, but they can't decide.

 

[Shar Dundred]

I still disagree with the entire "user reviews can hope to get highlighted" thing.

If you already remove reviews, remove them entirely.

 

[IcemanBo]

Chaosy, your thughts make no sense. I'm not sure what you are talking.

Shar, no offense, but have you read the thread? It's not about removing reviews at all. They still remain valuable and essential for approval, just as they were before.
Only the rating from reviewers is combined with ones by community.

 

[Ralle]

I never said we are removing reviews. We are removing review ratings. This means that all star ratings will be unified.

Later on I revealed that we are going to get back an analog of "moderator review" but having the ability to have multiple of them. Right now moderator/reviewer reviews are just comments and they are not easy to find. Tagging them as "review" means that they will show up in the top like the old days. They are not to be confused with ratings.

I also said that normal user posts MAY be marked as review. I am adding this option because I know for a fact that moderators once in a while find a review that they couldn't write better themselves, so this eases that workflow. Not to mention that it's probably pretty cool for a user to get his/her comment tagged as a review.

 

[Aleister]

How old are you again to think you have the right to call anyone "kid"?

Facepalm... it's an expression!

For example the Oscars which included but is not limited to art is decided by vote of 5000-ish people (as of 2012, it may be more now)
Source: Academy Awards - Wikipedia, the free encyclopedia

Correction, are 6000-ish people.
Anyway, f4ck the Oscars, f4ck the Emmys, f4ck the Grammys, f4ck all that sh!t.
There are a lot of films that are winning the Oscar, but they are complete rubbish.
The same is happening with the music awards, and other arts.
Paradoxically, there are amazing movies that haven't even been nominated for some award.
So, it has nothing to do.

 

[Kazeon]

Chaosy, you're wrong. The moderator/reviewer will do as he/she has done so far. Write a review. Now it won't just be a post however. It will be shown in the top with the bundle. However, there is the ability to have multiple reviews. If two reviewers disagree, they can both write a review. Users can hope for their posts to be marked as reviews (by a reviewer/moderator) as aspiring reviewers, but they can't decide.

I like this idea very very much!
The more I think about the impact of this change, the more I understand how it will impact rather positively.
And if I try to explain my opinions any further, I will only ended up repeating the main post. Staffs must have decided this very carefully with a long list of considerations. Gj!

 

[Shar Dundred]

I also said that normal user posts MAY be marked as review. I am adding this option because I know for a fact that moderators once in a while find a review that they couldn't write better themselves, so this eases that workflow. Not to mention that it's probably pretty cool for a user to get his/her comment tagged as a review.

And on the other side, it might also encourage mods to rely on user reviews only instead of
even downloading the map. Wouldn't be the first time (example: Map Section 2014/15)
It could make them rely on certain users ( exampe: Map Reviewers group) opinion without
even checking themselves.

Again, I have no opinion about the entire "remove reviewer rating" thingie, but I disagree
with this particular plan of yours - even if it is only a consideration at this point.

Facepalm... it's an expression!

Doesn't change my point, does it, boy?

 

[Kazeon]

@Ralle:
Anyway, isn't it better yet to display reviewers and moderators ratings next to their reviews and display the average rating right below user ratings as usual? So someone may look at user rating to look for popular resources and look at official rating to look for quality resources.

And how about displaying every users' ratings below their comments, just like in Hive 1? I bet it's not an easy change tho.

 

[Chaosy]

Chaosy, you're wrong. The moderator/reviewer will do as he/she has done so far. Write a review. Now it won't just be a post however. It will be shown in the top with the bundle. However, there is the ability to have multiple reviews. If two reviewers disagree, they can both write a review. Users can hope for their posts to be marked as reviews (by a reviewer/moderator) as aspiring reviewers, but they can't decide.

Ah, so a post can only be converted to a review (by staff).
I was under the impression of an extra button where the user could post a review whenever.

The submission rules wont change then?
Because I highly doubt any user will review a submission fully from the submission rules' standpoint. Thus marking a post as a review maaaaay be a rule breaker.

 

[Murlocologist]

And on the other side, it might also encourage mods to rely on user reviews only instead of
even downloading the map. Wouldn't be the first time (example: Map Section 2014/15)
It could make them rely on certain users ( exampe: Map Reviewers group) opinion without
even checking themselves.

Mods/reviewers of art/visual resources would quote valid members comment with agreement as verdict to resource's quality when there would be such comment. Which is nothing new and is common sense to be done since what you see is what you get. However, doing this on maps or models without testing them on your own would surely be a very reckless idea.

Thing I dislike the most is that more bullshit resources will come as 5/5 ones, rated as such by uploader's pals, no one will know the true (or at least somewhat objective) quality of resource before they download it, because no one got that much time to read what mod says as he could have simply visualized his opinion by 1, 2, 3, 4, or 5 stars. If every resource would get 10 times more than 4 ratings by randoms than I guess we could believe rating system. But back in day of reviewers rating, who REALLY gave a damn how 4 random commentators rated it with 5/5 when mod rejected it. Did you realize how many 5/5 resources lay in substandard, not even approved by mod? In light of that, where mod=random user, and rating in one, we could start to doubt how justifiable is to put 5/5 resources in rejects section based on review from one person.

 

[Deleted member 219079]

Too long have we silently spectated the horrendous system that can only be compared to being in dirty pens of unhygienic, cold barns without drinkable water and nutritious food, lead by narcissistic leaders who don't care for our rights. We are the most important part of any community, the users; we are the very reason this site has got so far. We are all equal! To demonstrate the power of an individual and further disprove hierarchy in modern hive, we rid ourselves of the misleading and subjective practice that is moderator ratings. We will further be equal by promoting good user's review, thoroughly constructed production of free, liberal mind, which better describes our opinion than a cold statement written as a part of a systematic protocol, spawn of a corrupted mind brain washed into justifying the acts of the system. No longer will we be stuck to our history of grouping members of the community into filthy castes, as if we were destined to be of certain tier. We are born just as the great leaders of today were born. This is only the first step, as there's a slew of improvements to come! We won't stop until the perfect equality of each user having the exact same privileges is attained. We know the times are rough and the social stratification is strongly present, but you can rely on our efforts in order to achieve brighter tomorrow and leave the practices of mistreatment behind! This is the time of change and justice that will be written in history books as the most important movement of man on Earth. Vive la liberté!

Hidden

Edit: Holy shit I just got invitation to the staff by all the admins and all the mods recommended me, should I say yes?

 

[IcemanBo]

Thing I dislike the most is that more bullshit resources will come as 5/5 ones, rated as such by uploader's pals, no one will know the true (or at least somewhat objective) quality of resource before they download it, because no one got that much time to read what mod says as he could have simply visualized his opinion by 1, 2, 3, 4, or 5 stars. If every resource would get 10 times more than 4 ratings by randoms than I guess we could believe rating system. But back in day of reviewers rating, who REALLY gave a damn how 4 random commentators rated it with 5/5 when mod rejected it.

If you really care what users and mods think about quality you should take the time to read the comments and reviews. Ratings are worse and less in detail than written and explained critiques.

Did you realize how many 5/5 resources lay in substandard, not even approved by mod?

This may happen from time to time, but doesn't make it less valid as before. Everyone has the rigth to rate, and people might not spot each flaw when they do so.
When a reviewer/moderator is about to make a verdict, has a detailed analyse, and spots a critical flaw, it won't be approved obviously, but still got a good rating from others (which is okay, too). A resource might be good, but not approved because of X/Y. Read the respective reviewer-comment, and then you should know the reason of it's state.

 

[Murlocologist]

This may happen from time to time, but doesn't make it less valid as before. Everyone has the rigth to rate, and people might not spot each flaw when they do so.
When a reviewer/moderator is about to make a verdict, has a detailed analyse, and spots a critical flaw, it won't be approved obviously, but still got a good rating from others (which is okay, too). A resource might be good, but not approved because of X/Y. Read the respective reviewer-comment, and then you should know the reason of it's state.

And yet flawed opinions from randoms got same weight as mod's.

If you really care what users and mods think about quality you should take the time to read the comments and reviews. Ratings are worse and less in detail than written and explained critiques.

No one is going to bother with that. Besides in this environment the reliable source reviewer's grade represented got pushed back in tier of random opinions. And I don't agree ratings are worse than written critique as in the end I don't care about details for which this may be bad or decent resource, but I want to see what is the ultimate verdict instead of reading how author should improve it.

 

[IcemanBo]

And yet flawed opinions from randoms got same weight as mod's.

Mods are gods or what? There are skilled people who are not in staff. And when you read the comments and you are not a total beginner, you can see or check it pretty fast im most cases if the review you read is just random crap or is reasoned.
And btw, why others' reviews should get less weight than a reviewer's review? All can give a review.
In the end anyways the moderator hold responsibility for what to make, or not.
And as Ralle said, not all random crap will be marked as review. I don't see the statement of this point.

No one is going to bother with that.

That's your problem then. Again, if you want to know something about quality, ratings won't help you to explain what point is good or bad. Instead, a written review might help more.

 

[Murlocologist]

Mods are gods or what? There are skilled people who are not in staff. And when you read the comments and you are not a total beginner, you can see or check it pretty fast im most cases if the review you read is just random crap or is reasoned.
And btw, why others' reviews should get less weight than a reviewer's review? All can give a review.
In the end anyways the moderator hold responsibility for what to make, or not.
And as Ralle said, not all random crap will be marked as review. I don't see the statement of this point.

I'm talking about ratings which all go to same pool now, not written critiques. Written critiques in my opinion are valid primary, if not only, to resource author, shouldn't be mandatory to read what reliable personas think to realize resource is high quality. And with merged ratings it seems to be pretty mandatory. I'll rather value mods opinion about resource if they give it 5 stars than rating from member who joined yesterday. Maybe you would do the opposite?

And btw, why others' reviews should get less weight than a reviewer's review?

Because mod is not appointed to that position to wear fancy rank icon or award, but to be a reliable source and someone whose opinion (imo primary rating) is not (at least transparently) biased by irrelevant stuff.

 

[Kazeon]

Mod ratings are actually just almost as unreliable as user rating to determine resource's quality. As have been stated in main post, hive couldn't retain consistent standard among different staffs within different times. It's just the moderator's/reviewer's personal scoring after all. So there is no point in defending it.
To solve this is by displaying good reviews atop of the resource thread, so you can judge quality by reading those reviews before downloading. By reading reviews you can acknowledge what's interesting/good/lacking about the resource, no matter when the reviews were written.

The challenge, however, is to have reviewers competent enough to write proper official reviews which are credible enough to be a view-point for judging a resource's quality. Most of them are just saying, "I think this one is well-done/lacking", etc in their reviews. No offense but those are not good enough, and frankly, not cool A good review should have told the reader about the general idea of the resource and what's interesting about it and what not. Just an example:
"ABC is a map about bla..bla..bla. (General description about the map from the reviewer's perspective)

ABC features x, y, z (explains the features of the map and its advantages)

However, when this review is written, it can use some improvements: (lists all reviewer's suggestions)"

The example could've been better but that's what I have for now. This way the review will remain valid no matter when the review is written. With a rating, such thing can't happen.
I know this part is out of place but I don't know where else should've I posted it. I won't bother starting a new thread neither.

 

[Murlocologist]

Mod ratings are actually just almost as unreliable as user rating to determine resource's quality. As have been stated in main post, hive couldn't retain consistent standard among different staffs within different times. It's just the moderator's/reviewer's personal scoring after all. So there is no point in defending it.

That is unfortunately wrong, because sample of 2-3 user ratings (how much is there usually per resource) tells absolutely nothing about resource in 90% of times. Mod rating is sometimes wrong or biased or personal, however in most of cases rating is shaped by criteria of quality standards website put up.

 

[Kam]

I understand the desire to level the field between mods and users but there needs to be consideration for the fickleness of people. Mods have written guidelines and standards to work from. I've been talking to the staff at ModDB for some time about their mod rating system. Currently anyone can make an account and issue a 1 or 10 with no written response and it counts as much as a real user who took the time to play and write a review.

 

[Kyrbi0]

And on the other side, it might also encourage mods to rely on user reviews only instead of
even downloading the map. Wouldn't be the first time (example: Map Section 2014/15)
It could make them rely on certain users ( exampe: Map Reviewers group) opinion without
even checking themselves.

And that's... bad? If a user test a map & gives a well-thought-out, meaningful review, what's wrong with a Moderator nominating it as an 'official review'?

I understand the desire to level the field between mods and users but there needs to be consideration for the fickleness of people. Mods have written guidelines and standards to work from. I've been talking to the staff at ModDB for some time about their mod rating system. Currently anyone can make an account and issue a 1 or 10 with no written response and it counts as much as a real user who took the time to play and write a review.

Do they? I don't know that they do, here.

 

[deepstrasz]

And how about displaying every users' ratings below their comments, just like in Hive 1? I bet it's not an easy change tho.

That kills the anonimity I don't like. Some backbone is needed. Might encourage users to not rate in a big number so better leave it anonymous, I guess.
Anyways, killing moderator/reviewer ratings is useful unless we get a jury.

Because I highly doubt any user will review a submission fully from the submission rules' standpoint. Thus marking a post as a review maaaaay be a rule breaker.

Mods/revs should only fortify their own critiques with what other users comment.

However, doing this on maps or models without testing them on your own would surely be a very reckless idea.

Which is very possible, actually prone to happen, maybe not with the current people but surely in the future. The world we live in is proof of that.

Did you realize how many 5/5 resources lay in substandard, not even approved by mod?

This is quite a strong point though not without err since mostly usually one moderator acts for one resource. Rarely have I seen another moderator/reviewer cross the line and change the outcome of a map to rejected or accepted after first being moderated by another privileged user.

we are the very reason this site has got so far.

Not all of we since mostly Ralle and other few people that (donated) actually were the ones to keep it alive all these years. Sure, creation is most important and the purpose of this site but, without the foundation...

We will further be equal

Who does actually believe that illusion?

No longer will we be stuck to our history of grouping members of the community into filthy castes, as if we were destined to be of certain tier

Here's for equality:

Spoiler

Mods are gods or what?

They are experienced people in the field. Or at least it's what they should be.

There are skilled people who are not in staff. And when you read the comments and you are not a total beginner, you can see or check it pretty fast im most cases if the review you read is just random crap or is reasoned.

Those skilled people you're writing about do not invest the same amount of time a mod/rev does because they don't care that much about doing so. They usually report stuff that they're pissed at. Others, fewer, suggest improvements but you'd have to agglutinate many user comments to get close to a review that a moderator/reviewer is dedicating time to. Of course, things can be missed in a review but that doesn't make it equal to random users writing stuff.

That's your problem then. Again, if you want to know something about quality, ratings won't help you to explain what point is good or bad. Instead, a written review might help more.

A written review with facts verified by an experienced person, preferrably a moderator/reviewer.

Mod ratings are actually just almost as unreliable as user rating to determine resource's quality. As have been stated in main post, hive couldn't retain consistent standard among different staffs within different times. It's just the moderator's/reviewer's personal scoring after all. So there is no point in defending it.

Sounds like one of your resources got rejected and you didn't like it. Rude or not, the thing is, we're talking about experienced persons. That's why not just anybody should be appointed as a mod/rev.

A good review should have told the reader about the general idea of the resource and what's interesting about it and what not. Just an example:

Orly? Why don't you invest time in what's supposed to be a philanthropic hobby if you know how to do it better?

Currently anyone can make an account and issue a 1 or 10 with no written response and it counts as much as a real user who took the time to play and write a review.

And that's good?

 

[Rufus]

Thinking about moderators "highlighting" other users reviews:
I think this will be problematic. Sure, people that are potential downloaders will read it and find it useful. However, when the author updates the resource, the comments will remain. This means that not only do we have to check through an entire resource again, (which can be really frustrating in itself) but we also need to unhighlight and highlight new reviews. I think this will be messy. Also, the author might not even be happy with the review being highlighted in the first place. What do we do then?

I will probably not use this system if it goes live.


However, I do support that the moderator comment should be on top of the comments, because some "expert" input is good.
The rating system is kind of broken as it is now, as most resources end up getting 4 to 5 stars no matter the quality, and therefore, some "expert" input, conveniently available at the top of the comments is very useful.

Furthermore, I think the name of the moderator should be next to the review, instead of just "moderator" as it is now. This will give the reader an image of who checked the resource, and it will be easier to know if you are likely to like the resource or not, depending on if you have agreed with the moderators ratings in the past.

 

[Kyrbi0]

Thinking about moderators "highlighting" other users reviews:
I think this will be problematic. Sure, people that are potential downloaders will read it and find it useful. However, when the author updates the resource, the comments will remain. This means that not only do we have to check through an entire resource again, (which can be really frustrating in itself) but we also need to unhighlight and highlight new reviews. I think this will be messy. Also, the author might not even be happy with the review being highlighted in the first place. What do we do then?

What stops the moderator from just 'appending' the highlighted user review, as they currently do with their own (strikethrough the original text & append more)? And even if they can't, just post another review right underneath it and say "In addition/appended to the above^^, blah blah blah".

'Authors not being happy with reviews'? Say it ain't so. ; )
Such is life. We cannot choose how people react to our work.

 

[Kam]

@deepstrasz

No, it's not. It's terrible. I am pointing out that our new path is heading towards that kind of system.

 

[Remixer]

Hmm. Technically the reviewer's rating has no major impact on where the map goes and how it acts, so the consequences of removing it wouldn't be too drastic I guess. What matters or at least should matter the most is the review comment, not the final rating itself.

Even if the actual reviewer's rating is removed, I assume we can continue to rate the resources, but not specifically included in the approval of the resource, so it would be up to the author whether he values the mods/reviewer's rating more than the others.

 

[Kyrbi0]

Hmm. Technically the reviewer's rating has no major impact on where the map goes and how it acts, so the consequences of removing it wouldn't be too drastic I guess. What matters or at least should matter the most is the review comment, not the final rating itself.

Even if the actual reviewer's rating is removed, I assume we can continue to rate the maps, but not specifically included in the approval of the resource, so it would be up to the author whether he values the mods/reviewer's rating more than the others.

Precisely.

We gotta separate "rating" from "review" in this conversation.

 

[Kam]

Precisely.

We gotta separate "rating" from "review" in this conversation.

Agreed. My issue and point in mentioning ModDB is that there needs to be some requirements to rate that are applied to everyone. No new accounts. Might we do an expansion on the reputation system that brings up a text box in order to rate with a word minimum? In that case if there were questions on validity later there is some metric to judge.

 

[The_Silent]

What about contests, they seem to work with the exact opposite philosophy? Which is why judges ratings counts for more than votes. Seems to be a strange discrepancy in hive policies now.

 

[GhostThruster]

because there's a discrepancy between contest judging and the moderation process - they're different entities. please don't make me go into specifics, and break down why your logic there is flawed.

 

[deepstrasz]

Can we then have rating enabled for resources only after they are approved and not in the pending or awaiting update state?
Yep, that means eff the older maps

By the way @GhostThruster, get an avatar!

 

[Shar Dundred]

The previous posts in this thread give the impression that this "user review highlighting"
hasn't been discussed - maybe not even mentioned at all! - with the staff or the reviewers
before being published here.

Don't mind me, just pointing out a few things here...

 

[Rui]

So, hello everybody. How've you been?

Before I answer a few of the points made here, I'll address the matter at hand — hopefully keeping this post short.


Pros of this decision:

☑ Removes moderator hegemony. Several have defended that one's opinion should not be definitive, me among them [1].
We've heard reports that moderators were sometimes unfair with their ratings, such as with icons [2][3] or misguided in their criteria, such as with spells [4].
Standing by that principle, as a map moderator, I'd embed my testing crew's opinions in the moderator rating [5][6].

​

Pitfalls:

☒ We're asserting that moderator rating should go because moderator opinion is: 1) biased; 2) inconsistent over time, and 3) not superior to others. However, we're giving moderators the ability to tag posts as reviews at their discretion. Aren't we then subjecting to the very same flaw? we just tried to eliminate?

☒ As @SpasMaster just rightly warned, moderator rating clearly distinguished user criteria (e.g. based mostly on hype and popularity, in the case of Maps) from Hive criteria (based on modding aspects of the game). Regular users tend to rate at an end of the spectrum—either 1/5 or 5/5—and their comments are hardly reviews: "5/5 very cool". A recent critique on this regard by @Murlocologist can be found here [7].

☒ As mentioned here, some users were appreciating moderator feedback, such as @Chaosy and @SpasMaster.

(?) What happens to Director's Cut, if it is based on moderator rating?
​

My proposals:

Spoiler

○ We cannot assert consistency and unbiased reviews with a centralized judge. Because users favor mostly popularity while ignoring other important modding criteria, they also mustn't be entrusted with this task entirely. Therefore:

1. Senior users (rough definition below) should designate which posts should be reviews.
2. A post is a review only if it has a minimum number (that number is up for debate) of designations.


NOTES
» How to appoint these senior users? An idea would be to manually appoint the people currently attending the several sections. Or through post count and reputation (with the usual controversy that comes with it ).
» In any case, to make the process automatic for the future, a user who has had a minimum number of his posts marked as reviews becomes a senior member.

​

○ Replace moderator's rating with a senior member rating, that is calculated from posts marked as reviews.

ARGUMENTS IN FAVOR OF THIS
» The desire to see special members as the jury for comments and ratings remounts as far backwards as a proposal by @Norinrad in 2005 [8].
» Plus, it was also the wish of over 70% of the Hive community several years ago [9]. Probably still is.
» Although we had the Reviewed status, it became broke at some point for unknown reasons [10].​

 

[deepstrasz]

Regular users tend to rate at an end of the spectrum—either 1/5 or 5/5—and their comments are hardly reviews: "5/5 very cool". A recent critique on this regard by @Murlocologist can be found here [7].

To not necropost that thread I'll write here that not everybody is a critic and most people here are kids. More than that, not just anyone has an insightful mind to be able to give arguments in many fields.

Senior users

Privileged users like mods, revs and admins?

2. A post is a review only if it has a minimum number (that number is up for debate) of designations.

Does that mean more than 1 mod/rev have to mark a post as a review? That basically means a mod/rev jury which can instead be used for the resource itself.

How to appoint these senior users? An idea would be to manually appoint the people currently attending the several sections. Or through post count and reputation (with the usual controversy that comes with it

Oh... doesn't that basically turn those users into reviewers, a rank we already have? But now, it will be given to people that won't necessarily have anything to do with actually creating stuff for Warcraft (basically critics)?

Anyway, not a bad idea at all. It might just work.

 

[Kyrbi0]

Nice, well-organized thoughts, Rui.

I wonder if the Rating system could be made to mirror the Rep system; basically it would auto-provide a text box that had to be filled or else no Rep Rating could be given.

 

[deepstrasz]

I wonder if the Rating system could be made to mirror the Rep system; basically it would auto-provide a text box that had to be filled or else no Rep Rating could be given.

Will you elaborate please? I don't really get it.

 

[Kyrbi0]

Will you elaborate please? I don't really get it.

Um... I'm really not sure how much more simply to put it.

How is Reputation given? You see something you like, so you click on the "Give Reputation" button. A text-box appears. If you fill it out (i.e. give a reason for the Reputation you've given) & click "OK", it gives the Rep & your explanation appears on their profile. If you don't, you can't give the Reputation.

Now replace every instance of "Reputation" with "Rating" and that's basically it.

 

[deepstrasz]

Now replace every instance of "Reputation" with "Rating" and that's basically it.

Oh, thanks for that.
Who exactly is supposed to be able to do that? Anyone? I mean how would that be much more different than writing a comment and rating from 1-5? I mean people could just spam anything there even a dot. I'm not sure how this would be objectively better.

 

[Chaosy]

However, we're giving moderators the ability to tag posts as reviews attheir discretion. Aren't we then subjecting to the very same flaw we just tried to eliminate?

Nice catch @Rui
That is quite true.

 

[GhostThruster]

it's not true tho. and we're not trying to eliminate, but rather minimise. there is no system that can completely eliminate the key issues pointed out.

tagging multiple posts as reviews holds much more objectivity than a single mod reviewing a resource. this only falls apart if the mod in question isn't impartial (e.g. he only tags reviews that agree with his assessment), in which case said person shouldn't be a mod in the first place.

reviews also aren't quantitative, so they don't suffer nearly as much as a rating does over time when communicating the quality of the resource to a user. yes, maybe reviews will contain outdated suggestions/bugs, but anyone with eyes and an understanding of calendars should be fine with it. furthermore, reviews aren't meant to be a potential search filter, unlike ratings. hence, the need for them to be consistent over time is even less.


rui, your proposal is basically the current plan, except worse. as it is now, reviewers like myself are selected based on expertise, not automatically granted the ability to designate reviews based on some arbitrary definition of a 'senior member'.
multiple designations is a good idea, in theory. but is the hive really at a point in its life cycle where resources get multiple reviews, and there are an abundance of 'senior members' that read said reviews? and do you really believe a single moderator/reviewer cannot objectively decide upon which posts to be tagged as reviews by themselves, especially if we establish guidelines for review-tagging?

 

[Chaosy]

this only falls apart if the mod in question isn't impartial (e.g. he only tags reviews that agree with his assessment), in which case said person shouldn't be a mod in the first place.

This was how Ralle said it would work.
A mod would make a user post a review IF he had the same thoughts.

Imagine this.
I test a map for five minutes. I look at the reviews and find one I think fits my thoughts and the rating seems fair. I mark it as a review.

 

[GhostThruster]

i think ralle was just raising an example on one possible approach a mod/rev might take. not well thought out or phrased - it's easy to interpret "fits my thoughts" as "agrees with my assessment completely", but it's not the case seeing as ralle immediately mentions afterwards "and the rating seems fair".

i really don't think he means:

Imagine this.
I test a map for five minutes, think it's worth 4/5. Another user tested it for 10 hours, and wrote 4 intricate paragraphs detailing the flaws of the map. He rates it 3/5. Because his opinion differs from mine, I won't mark it as a review. Power abuse ftw