Removing Review Ratings

[Rui]

Privileged users like mods, revs and admins?

Yes, but a user could become senior under other criteria (like I suggested: at least X of his posts were tagged as reviews; reputation; post count).

Does that mean more than 1 mod/rev have to mark a post as a review?

Yes.

Oh... doesn't that basically turn those users into reviewers, a rank we already have?

I've been out of the loop and haven't caught up with everything that's happened. I do not know exactly how reviewers work right now.

But now, it will be given to people that won't necessarily have anything to do with actually creating stuff for Warcraft (basically critics)?

Like I've said once, you do not need to know how to cook to be a cuisine critic. EDIT: Still, I'm not sure how good a review it can be if it doesn't cover modding aspects of the game. "Good" by our standards, at least.

---

it's not true tho. and we're not trying to eliminate, but rather minimise.

The original message was actually «subject to the same flaw». I added «we just tried to eliminate» only to make my post more comprehensible (wary of people not understanding).

tagging multiple posts as reviews holds much more objectivity than a single mod reviewing a resource. this only falls apart if the mod in question isn't impartial

I agree, but if the mod was impartial, then he should review properly/impartially. I understand what you mean: by not reviewing directly, chances are that designated reviews differ in-between them enough to cover a broader range of perspectives. Although I recognize a mitigation to it, the core issue is still there, and I can already imagine people complaining «why does mod X think Y's post is more a review than mine is?». Because a wider perspective is opened with senior members instead of a single mod, the more unlikely this quarrel becomes.

Spoiler

On the other hand, I wonder. To what degree is it possible to be unrecognized by a majority of seniors and have a post that's a "good review". Impossible is too strong a statement.

maybe reviews will contain outdated suggestions/bugs, but anyone with eyes and an understanding of calendars should be fine with it.

Spoiler

Just a note. When reviewing maps, I considered it a good practice to point out what version I was reviewing.

as it is now, reviewers like myself are selected based on expertise, not automatically granted the ability to designate reviews based on some arbitrary definition of a 'senior member'.

The definition of senior was left purposely arbitrary with the intent of debating around it.

multiple designations is a good idea, in theory. but is the hive really at a point in its life cycle where resources get multiple reviews, and there are an abundance of 'senior members' that read said reviews?

Is the Hive at a point in its life cycle where a single moderator is expected to take on the entire load of reading every post and tagging it as a review (or not)?
My answer to your question is: yes, if we don't set the requisite too high (just 2 or 3 designations would already be good enough) and no if we set that bar too high.

and do you really believe a single moderator/reviewer cannot objectively decide upon which posts to be tagged as reviews by themselves, especially if we establish guidelines for review-tagging?

Objectively? That opens doors to philosophical debate.
Sure, you can turn it more or less objective with review guidelines (provided they are objective themselves), so long as they're not too restrictive. I find a mod's work as excessively bureaucratic in that case. Since I've found a number of people who enjoy reading reviews, who knows (albeit the job at hand is slightly different).

EDIT: Oh man, that's a large post. D;

 

[BLazeKraze]

I'm agree by removing moderators review ratings since their taste, perspectives, and art style might be different to other people. So let people decides on every resources. Some moderators might be lenient, or the others are always giving high ratings.

 

[deepstrasz]

I've been out of the loop and haven't caught up with everything that's happened. I do not know exactly how reviewers work right now.

Well, they're basically Map Moderators. They can accept/"reject" maps (move them in places).
It'll be tough to get more than 1 mod/rev per resource section. The Maps section has 1 mod and 2 revs from what I know. That's quite something. If they all are supposed to "rate" user reviews so they can then become senior users who themselves would count as revs (but not as ones to also mark reviews?), things would get complicated. It's basically better if those 3 privileged users of the Maps section would just form a jury for every map instead.

 

[GhostThruster]

I've been out of the loop and haven't caught up with everything that's happened. I do not know exactly how reviewers work right now.

well this is basically your senior members idea, except we're not automatically chosen, we're chosen on expertise in our respective resource sections.

@Rui i see that you take the kyrbi0 approach to discussions - good intentions and some good ideas, but your solutions are needlessly complex, almost like you want to discuss for the sake of discussion rather than practicality & action. i wish i cud indulge you, but i dont have the time these days, certainly not to address you point-for-point. so lets cut to the chase.

we're debating the extent to which we need to mitigate mod/rev inconsistencies and subjectivity. yes, allowing an abundance of 'senior members' to designate reviews does further mitigate rating/review flaws. but at a point it just becomes stupid bureaucracy and entangling of the review process. like we can go on and on and suggest we have 'super mega senior members' that designate whether a 'senior member's designation of a review is valid, but it's not necessary. you have to consider whether your proposal will have a meaningful positive impact that's worth the complication, and whether it fits the current context of the hive (its users, its activity etc.). to me, a review requiring a single mod to tag it is good enough, and your proposal is unnecessary at best, and an overcomplicated system that solves nothing at worst.

 

[Kyrbi0]

@GhostThruster <3

 

[Murlocologist]

Well, both pro mod rating and anti mod rating sides should consider how active Hive is. And during whole debate this aspect to ratings' value has been overlooked. It is very simple, Hive is not that active to assign new mini mods, putting aside how concept of such is complex or not, Hive at the moment has no capacity for this. Activity of users in resources section can even less be attributed as active. Those few who give comments in resources either:
1. give actual review
2. spam with 5/5 one liners
3. got no idea what are they talking about
4. spam with off topic one liners.
5. are giving ignorant reviews
Not to mention that these above profiles of commentators most usually give rating for no resource quality relevant reason. In the end we have a load of crap rated as 5/5. And this matters for those who mods exist in the first place. Do mods exist to deal with each individual separately or to deal with whole community with both its members and its content? Obviously first is not the correct answer. No one can affect uploaders of resource, no one can order them to try to improve it. However encouragement to do so is always a good practice, and so is done in form or reviews. But this is relevant to uploader, not community or those who will download it. They rely on rating. Review presented in a form of a number which is essential to filter good from lousy. Good and lousy resources do not hold universal quality, they fall as such (good or crap) by the key of time they were made in.

Mods can't be constant over time, sure, 2006 and 2016 standards are different, but what you are trying to accomplish seems like washing your hand from any sort of judgment presented by people who are appointed to mod position to do give judgment. Give judgment, not based on subjectivity, bias or other, but standards of wc3 modding. And times change, quality of resources are not as some try to put it universal. Every resource was made in certain context: year, available tools, other resources' quality etc. And so were ratings given as well. I fail to see lack of validness of rating there.

 

[Rui]

@deepstrasz Newly appointed seniors would be able to mark posts as reviews, yes. Self-sustainability is the point.

@GhostThruster I lack the time too, but I needed a break from those damn papers and decided to make it worth it. And you're right in that I like to debate. Discussing stuff really shouldn't be viewed as such a bad thing. I'm okay with your stand though. I was just pointing out a logical flaw in the thought process behind this decision.

Regarding my solutions, I don't view them as overly complex, but that's because in my uni course we solve problems for a living .

Anyhow, my real concern is for mod ratings. Senior ratings (or reviewer ratings, if you prefer) was an individual proposal of its own: I really do believe it should be implemented, replacing mod ratings.

 

[deepstrasz]

Anyhow, my real concern is for mod ratings. Senior ratings was an individual proposal of its own: I really do believe it should be implemented, replacing mod ratings.

And moderators would then only do what, mark reviews (that seniors will be able to afterwards) and decide the place of a resource? I mean, shouldn't the mods/revs also add their rating and review? I'm confused.

 

[Rui]

Call it reviewer ratings if you prefer. I'm saying there should be a replacement for mod ratings that is calculated from (EDIT) ratings given by reviewers posts marked as reviews.

 

[deepstrasz]

Call it reviewer ratings if you prefer. I'm saying there should be a replacement for mod ratings that is calculated from ratings given by reviewers.

That mean we still keep user ratings (basically keep both types of rating systems but change the specified one). Good. Basically it's an addon. What I'm saying is I don't want mods/revs to slack while the seniors do the actual/most work.

 

[fladdermasken]

What I'm saying is I don't want mods/revs to slack while the seniors do the actual/most work.

As I understand it, moderators will still do all they did before minus give the resource an official mod rating.

And if people are confused why mod ratings are removed, I wager it's partly because of the enormous amount of extra work that comes in the form of dealing with complaints from people who are pissed that they got a 3/5 when they think they deserve a 4/5 or higher.

In my experience most people think mod ratings are great, as long as they get the higher scores.

 

[Shar Dundred]

#RuiForPresident

 

[Kam]

In my experience most people think mod ratings are great, as long as they get the higher scores.

Agreed. The best way to avoid this is to insure that the review(s) are both concise and informative in where there needs to be improvement.

Edit:

For resources there should be a checklist of basic items all resources of that type need to have.

For instance for hero models:

-Proper death, decay animation
-Hero glow
-Portrait animations for speech

A lack of one of those would immediately lose a star.

 

[deepstrasz]

A lack of one of those would immediately lose a star.

Or not be approved? Since stars can be taken because of the animation looking bad or the skin on the model looking weird and so on.

 

[Murlocologist]

As I understand it, moderators will still do all they did before minus give the resource an official mod rating.

And if people are confused why mod ratings are removed, I wager it's partly because of the enormous amount of extra work that comes in the form of dealing with complaints from people who are pissed that they got a 3/5 when they think they deserve a 4/5 or higher.

That is absolutely normal occurrence whenever someone criticizes and judges on others' work. Judging here (Hive) in most cases was objective, without bias towards the submitter and by some known criteria, but of course as in everything, there are times when they screwed up rating completely. But is all this said enough reasoning to banish mod rating? No.

 

[deepstrasz]

But is all this said enough reasoning to banish mod rating?

The problem is that usually not more than one mod/rev rates a resource and that doesn't make it good statistics.

 

[Wareditor]

No more director's cuts... Little by little the old Hiveworkshop is fading away

 

[deepstrasz]

No more director cuts... You might as well remove

The way I see it (in case it wasn't done this way), is for Director's Cut awards to be given to resources after all moderators and reviewers of said resource field gather and decide.

 

[Flux]

Director's Cut is useful for me to browse excellent models/icons which I could use.

 

[Kam]

DC is useful and needs to stay. It provides a metric for artists to see what a top tier product is.

 

[Murlocologist]

My view: Director's Cut

 

[Chaosy]

@Kam @Flux
Uh
But if DC stays.. why not 5/5 mod rating?
It does the same thing:
-Good to filter out worse submissions (unlike user rating which is *** ****)
-is decided by the staff rather than the user

 

[Kam]

DC is more than 5/5. It's for artists who go above and beyond any and all standards.