Contest judges

[Kyrbi0]

Hm, that might be a wise distinction to make ("punishment for rule breaking" vs "punishment for not meeting a contract").

Is there any discussion to be had on "how well a Judge should/must follow the prescribed Criteria decided upon in the Contest"?

 

[KILLCIDE]

Is there any discussion to be had on "how well a Judge should/must follow the prescribed Criteria decided upon in the Contest"?

That's all on the host. The host should read the reviews made by judges and decide what should be done before posting the results.

 

[deepstrasz]

That's all on the host. The host should read the reviews made by judges and decide what should be done before posting the results.

That, if the host has any expertise in the said contest's theme. If it's only about rules, mostly anyone can report if any improperness is made manifest.

 

[Murlocologist]

Punishment as -rep would happen only when judge would drop from the contest without notifying host that he will. In reality that would be: after a month after end of poll he has no wip of judging to show to host.

 

[Kyrbi0]

That's all on the host. The host should read the reviews made by judges and decide what should be done before posting the results.

Hm...

I suppose that could work, but I see two issues with it:

1) We already have enough of an issue with Judges taking a good deal of time in providing Judging; how well do you imagine things would look (or how badly would the Host be made to feel) if they said "ehhh, not good enough/fitting to Criteria; try again!". :<

2) In the situation of a Contest being Hosted by a Contestant (*cough*), there's a definite possibility for conflict of interest/abuse of power. A Host could continually claim the Judging was done incorrectly when really it was just them being graded poorly. Lots of drama to work through at that point.

~~

I guess what I'm suggesting is that there should be, somewhere (i.e. either written into the Contest Rules or perhaps even on the Site's Rules), an explicit statement about Judges following & using the decided-upon Criteria in their Judging. I did something like this in the 8th Hero Contest I ran with the line

• Judges will grade the final (up to) 5 heroes points, and must use the provided Criteria without modification

 

[KILLCIDE]

We already have enough of an issue with Judges taking a good deal of time in providing Judging; how well do you imagine things would look (or how badly would the Host be made to feel) if they said "ehhh, not good enough/fitting to Criteria; try again!". :<

I have no issue with waiting for good and fair reviews from the judges. I'm actually confused why there is an issue to begin with considering the results will not determine life or death for anyone.

In the situation of a Contest being Hosted by a Contestant (*cough*), there's a definite possibility for conflict of interest/abuse of power. A Host could continually claim the Judging was done incorrectly when really it was just them being graded poorly. Lots of drama to work through at that point.

That's what a moderator is for.

 

[IcemanBo]

If there's any problem with the judges which can not be solved between users/host/judges, the moderator who is resposible for the contest is the person to contact.
A host by himself is not really in position to override judge's opinion, or allowed to define a own rule-set for the judges to follow.

 

[Kyrbi0]

I have no issue with waiting for good and fair reviews from the judges.

Then this might not be the right thread for you. ;P
j/k

I'm actually confused why there is an issue to begin with considering the results will not determine life or death for anyone.

I don't have a problem waiting some time for 'good & fair reviews' (let's just assume we get them, for sake of argument). But 'some time', while admittedly subjective, does go to say that I think there should be an upper limit. I would not want to wait 100 years. I would not want to wait 50. I would not want to wait a year. 6 months... I would not enjoy it but I have seen it happen for what I consider to be good reasons.

It's that line that we are talking about. At least in part.

That's what a moderator is for.

And if that guy happens to be an involved Contestant? Even better. :<

It's true there are systems/people in place to resolve problems. But I am of a mind (& I think it's a wise policy) to do what we can to prevent problems from happening; craft Rules & Guidelines & expectations such that corruption & error don't have a place to grow (as much as possible). As they say, "far better to prepare & prevent, rather than repair & repent".

 

[KILLCIDE]

Then this might not be the right thread for you. ;P

It isn't, which is why I've only said this:

If failing to judge entries has punishments, then being a judge in the first place will have to be more rewarding.

I would not want to wait 100 years. I would not want to wait 50. I would not want to wait a year. 6 months... I would not enjoy it but I have seen it happen for what I consider to be good reasons.

If a contest takes over a year to post results, then it is an issue with both the moderator and host for not handling it properly.

And if that guy happens to be an involved Contestant? Even better. :<

The Staff will only allow a moderator of a contest to be a contestant if they trust the moderator. If you have an issue with their decision, you can always make a thread in Admin or Staff Contact.

 

[Murlocologist]

I should have made poll, but I knew I wouldn't be able to resist the sarcasm, so I didn't.

 

[Kyrbi0]

If a contest takes over a year to post results, then it is an issue with both the moderator and host for not handling it properly.

So it sounds like we can both agree that "a year+" is an inappropriate amount of time to wait. That's good. But that's one of the points of this thread; to determine if we can all publicly agree on a certain set of guidelines or expectations, rather than just leaving it up to personal, subjective & individual preference.

 

[deepstrasz]

2) In the situation of a Contest being Hosted by a Contestant (*cough*), there's a definite possibility for conflict of interest/abuse of power. A Host could continually claim the Judging was done incorrectly when really it was just them being graded poorly. Lots of drama to work through at that point.

That and imagine Hive friends: one a judge and the other a contestant and the host. True story. Problem is, there are few people for some contests (like music).

guess what I'm suggesting is that there should be, somewhere (i.e. either written into the Contest Rules or perhaps even on the Site's Rules), an explicit statement about Judges following & using the decided-upon Criteria in their Judging.

Yes, of course. But even there subjectivity is pretty high when there is only one judge for a contest.

I have no issue with waiting for good and fair reviews from the judges. I'm actually confused why there is an issue to begin with considering the results will not determine life or death for anyone.

Yeah, plus the judges don't take all the credit. The voters have a say too.

A host by himself is not really in position to override judge's opinion, or allowed to define a own rule-set for the judges to follow.

Let's just say the host is right one time and the moderator is not experienced in the field the contest is concerned.

---------------------------------------------------------------------------------------

Things to consider (not very helpful though):
-if a judge (or all judges) of a contest fail(s) to provide the final worksheet with the critiques then [after waiting a little time/trying to contact the judge(s)] the moderator (if experienced in the domain and not absent) may take the initiative for judging instead or
-the host may take the place of the judge(s) (if that person is experienced in the field; if the person also participated as a contestant, a sacrifice would save the whole deal assuming at least 4 contenders remain) or
-one of the contestants could turn himself into a judge (again with the condition that at least 4 contestants remain) and
-if that can't be done, we'll leave it at user rating (if there is a defined number to make it look like a wannabe statistic[!?])

Heck, what do we do if we get little to no voters? We extend the poll deadline but hypothetically, if nobody would be interested in voting on that contest, should we only leave the judging as the outcome to the ranking of the contestants?

And remember folks! Don't use judges in a text between ( ) with the s in between [ ] because you get all of your text stroke with a line.

 

[Murlocologist]

Some of you are talking about judges as if it would be immoral to have high expectations from people who are volunteers. And I am just gonna remind you that every poster in resources is, as well hobbyist. And yet there seems to be very strict rules around resources, they get turned down, get low rating, get bad review, get called rubbish and so on. We don't feel an imperative to be nice and considerate towards resource submitter, but we do when we talk about judges. What is the difference really? When did someone criticize a judge for delivering substandard/too simple review? Why would it be immoral to say to a judge: you made a bad review here an here? Why aren't you being humble to resource author in the same degree as you are towards a judge? Why would judge's time be more precious than the time people spent on making resources YOU CAN DOWNLOAD AND USE FOR FREE? Or maybe people are so considerate to judges in order to get more sympathy from them for next contests, I really don't know. Either way, I stick to what I said, waiting for results for anything more than one month after poll concludes is unacceptable. The judges are hobbyists, so is everyone else in here.

 

[IcemanBo]

At first, a judge won't be probably approved if he wasn't good in the respective judgement field.

But then, you're for sure right that reviews should have some quality. It should be ensured by the host, if he has enough expertise, and/or the appointed moderator.
The reviews should be read before posted, and when there are critical flaws there might be some need of discussion or changes at first, before the result publishment, depending on the case.

By the way. Much points I read here in thread are common sense, and not each obvious point must be ruled out.

 

[Kyrbi0]

...We don't feel an imperative to be nice and considerate towards resource submitter, but we do when we talk about judges. What is the difference really? ... Why aren't you being humble to resource author in the same degree as you are towards a judge?

Um, I don't know about anyone else, but I do. All the time. I think it's highly important.

I suppose I can see your argument; however, it presupposes a level of equality between Users & Moderators. In many ways, though, like the Nazarites of old, they are individuals bestowed with greater power & faculties, while consequently they have certain expectations & responsibilities they must account for.

It is to be assumed that the process by which a Moderator is chosen (& maintained) is highly dependent on their skill, demeanor, and activity. They are 'set apart'. So a different treatment may very well be in order.

~~~

So a month for you, eh? That feels pretty short, somehow, but I can agree it would be ideal.

 

[GhostThruster]

Some of you are talking about judges as if it would be immoral to have high expectations from people who are volunteers. And I am just gonna remind you that every poster in resources is, as well hobbyist. And yet there seems to be very strict rules around resources, they get turned down, get low rating, get bad review, get called rubbish and so on. We don't feel an imperative to be nice and considerate towards resource submitter, but we do when we talk about judges. What is the difference really? When did someone criticize a judge for delivering substandard/too simple review? Why would it be immoral to say to a judge: you made a bad review here an here? Why aren't you being humble to resource author in the same degree as you are towards a judge? Why would judge's time be more precious than the time people spent on making resources YOU CAN DOWNLOAD AND USE FOR FREE? Or maybe people are so considerate to judges in order to get more sympathy from them for next contests, I really don't know. Either way, I stick to what I said, waiting for results for anything more than one month after poll concludes is unacceptable. The judges are hobbyists, so is everyone else in here.

firstly, high expectations from voluntary work != punishing people that do not fulfill those expectations. the latter is immoral, the former is not.
and what? maybe you're just particularly prickly, but the average user is nice and considerate unless the resource author has broken the rules.
furthermore, judges have been criticized for their reviews - two examples i can recall are ike_ike and apheraz_lucent, who've had their judging removed and/or replaced.

i agree, a month+ is unacceptable. but a judge that doesn't fulfill his duties (in a timely manner or otherwise) should be treated just like a user who uploads a poor resource; the resource is removed and sometimes the user is told to improve, but the user is not punished or demonized for actively trying to contribute to the betterment of the site in a way that doesn't offend anyone or break any rules.

maybe your point isn't for there to be punishment, maybe you're actually on the same stance as me - that there should be enforced standards for the quality and timing of reviews, that judges should be replaced/removed within a timeframe, that judges should be expected to communicate progress. but your passive-aggressive tone and bombardment of idiotic rhetorical questions certainly don't help anyone's understanding of your argument.

 

[Murlocologist]

firstly, high expectations from voluntary work != punishing people that do not fulfill those expectations. the latter is immoral, the former is not.
and what? maybe you're just particularly prickly, but the average user is nice and considerate unless the resource author has broken the rules.
furthermore, judges have been criticized for their reviews - two examples i can recall are ike_ike and apheraz_lucent, who've had their judging removed and/or replaced.

i agree, a month+ is unacceptable. but a judge that doesn't fulfill his duties (in a timely manner or otherwise) should be treated just like a user who uploads a poor resource; the resource is removed and sometimes the user is told to improve, but the user is not punished or demonized for actively trying to contribute to the betterment of the site in a way that doesn't offend anyone or break any rules.

maybe your point isn't for there to be punishment, maybe you're actually on the same stance as me - that there should be enforced standards for the quality and timing of reviews, that judges should be replaced/removed within a timeframe, that judges should be expected to communicate progress. but your passive-aggressive tone and bombardment of idiotic rhetorical questions certainly don't help anyone's understanding of your argument.

If judge can't finish in one month and have no good reason for that, should be replaced. If that judge waited for 30 days to pass and made no wip of judgments to show to host and made 0 conversations with the host, he would receive -rep. That's how I see it. But the thing people keep pestering me is focusing so much on that punishment point. It is just another point, like double posting is reportedly prohibited. I see rules about judging in very arbitrarily way, probably because they are. Precedents of judges being criticized is no significance if it is considered as something one should refrain himself from doing. And no, I never call a resource a trash if that's what you were implying. I also wasn't asking rhetorical questions. If I was I'd ask: why is it controversial for one to discuss the judges?

 

[deepstrasz]

When did someone criticize a judge for delivering substandard/too simple review?

Why would it be immoral to say to a judge: you made a bad review here an here?

I see but then, who would decide that, other judges? The moderator and or host experienced in said field as long as they are not contestants? How does it work exactly?

Why aren't you being humble to resource author in the same degree as you are towards a judge?

Because resource authors don't get negarepped, that's why.

The judges are hobbyists, so is everyone else in here.

Exactly, that's why we have to consider things judging by that and not implement "drastic" measures. But please, enough with the questions. What's the solution?

At first, a judge won't be probably approved if he wasn't good in the respective judgement field.

If we get by what @Murlocologist wrote, we can just not accept a substandard judge for further contests which will decrease the number of judges/people to be considered as judges for future competitions.

But then, you're for sure right that reviews should have some quality. It should be ensured by the host, if he has enough expertise, and/or the appointed moderator.

Assuming they're incorruptible [if they know the judge(s)] and don't act like spoiled brats if they're also contestants.

furthermore, judges have been criticized for their reviews - two examples i can recall are ike_ike and apheraz_lucent, who've had their judging removed and/or replaced.

I'd love to see that.

i agree, a month+ is unacceptable. but a judge that doesn't fulfill his duties (in a timely manner or otherwise) should be treated just like a user who uploads a poor resource; the resource is removed and sometimes the user is told to improve, but the user is not punished or demonized for actively trying to contribute to the betterment of the site in a way that doesn't offend anyone or break any rules.

^This with the exception that judges won't be able to provide their critique again like resources are reuploaded or updated.

 

[Murlocologist]

I see but then, who would decide that, other judges? The moderator and or host experienced in said field as long as they are not contestants? How does it work exactly?

I am talking in general obviously. I wasn't suggesting to form a committee to decide if judge gave lousy review. It is common sense, it is just a matter of how people (members, posters, mods, randoms and others) react when judge actually does give a shitty review.

Because resource authors don't get negarepped, that's why.

Neither did the judges.

Exactly, that's why we have to consider things judging by that and not implement "drastic" measures. But please, enough with the questions. What's the solution?

Drastic? Well maybe drastic for someone who gives +rep 24/7, but for me -rep is far from drastic measure. What is the solution? Less arbitrary rules and a common sense.

 

[deepstrasz]

Drastic? Well maybe drastic for someone who gives +rep 24/7, but for me -rep is far from drastic measure.

The word was between "s. The only two things since Hive 1.0 that were a case of negarepping were the breaking of rules and loss in the Challenge Section.

What is the solution? Less arbitrary rules and a common sense.

Great. So many arguments.
Anyways, hopefully we're not debating without action.

You should call in the administrators as they will decide. Silent votes are not nice.