Contest judges

[Murlocologist]

If judge shows no work in progress on judging in one month time after the end of poll he should be replaced by different person. In addition, rule should say that if one has not got time to deal with judging, than he should not accept to judge. In addition, if judge realizes he won't make it in one month time, he needs to notify host so he can be replaced by different person. In addition if he goes inactive without telling host he will not be able to finish his task he should be replaced by different person and receive -rep.

 

[Kyrbi0]

Can you publicly state what the cause for this stance is? Like, what motivates you (or someone who agrees with you; one) to adopt this hard-line stance? Is it a matter of respect, a distaste for authority, a desire for efficiency, what? (I know this sounds trite but I believe every Rule should exist for a Reason, and every Reason should serve a Purpose; the "no cheating" Rule exists in order to stop cheaters, and that fulfills our purpose of an "ethically-grounded community" with true "tests of skill".)

Should this be determined on a Contest-by-Contest basis, or on a site-wide basis?

(I love the new Drafts system; I just closed my browser a few hours ago without saving, and my computer had a black-out as well... And it's all here. : P)

Personally, I think a lot can be taken care of if we mandate "communication" to a greater degree than currently. It's a big difference between a Judge simply 'disappearing without a trace' and a continual stream of information as to their progress & situation.

I'm a lot more likely to be lenient if the Judge himself is posting; stuff like "Hey guys, I'm 1/4-way through" -- "Hey there, had a busy week, only 1/2-way through" -- "Almost done, hopefully this weekend..." -- "Hey, lost my job/car broke down/death in the family; I'm gonna be a bit/let's find someone else/etc". Communication is more than half the battle, in my mind.

I'm not sure I'm entirely convinced, though, that we should neg-rep Judges. We only just began offering them plus Rep for performing the essential Judging service; seems like a slap in the face for what is ultimately a hobby (granted, the Contestants are also hobbyists...).

(Although you didn't talk about it here, but elsewhere,) I'm also not sure I can entirely get behind the "give up on Judges & go to Poll results"; at least, not without a drastic overhaul to the Polling system/requirements/expectations/zeitgeist/paradigm/phenomenon/bigwordshere.

 

[Murlocologist]

I'm not sure I'm entirely convinced, though, that we should neg-rep Judges. We only just began offering them plus Rep for performing the essential Judging service; seems like a slap in the face for what is ultimately a hobby (granted, the Contestants are also hobbyists...).

Negative rep is the only penalty you can reward judges with when they abandon task they took. Yeah, they are hobbyists, just like everyone else here is, and still we got punishment system, such as -rep, for a reason. Going inactive without notifying host, when you got a task to complete, is as much of misbehavior as spamming in thread is. Therefore, penalty (-rep) is perfectly justified.

(Although you didn't talk about it here, but elsewhere,) I'm also not sure I can entirely get behind the "give up on Judges & go to Poll results"; at least, not without a drastic overhaul to the Polling system/requirements/expectations/zeitgeist/paradigm/phenomenon/bigwordshere.

Oh if I ever had said we should rely on poll only I must have been drunk.

 

[deepstrasz]

No negarep. That's not just harsh it's uncalled for. This is not the challenges section! Things can actually occur and a judge or contestant can just drop out. Should we negarep contestants too for dropping out? NO! FTU! I think to avoid time wasting, the remaining judges should be taken into consideration. If only one judge was decided for said contest, then either wait for that person or find someone else instead.

 

[Kyrbi0]

Negative rep is the only penalty you can reward judges with when they abandon task they took. Yeah, they are hobbyists, just like everyone else here is, and still we got punishment system, such as -rep, for a reason. Going inactive without notifying host, when you got a task to complete, is as much of misbehavior as spamming in thread is. Therefore, penalty (-rep) is perfectly justified.

Hm.

Oh if I ever had said we should rely on poll only I must have been drunk.

Hm. I mean, it makes sense in the "lets finish & get results NAO" sense... But anyone can look back into the past few years of Contests & see that the Polls had serious issues. I would expect Multi-Vote polling at the absolute minimum, and preferably a Preferential Voting method (MBC, Schulze, etc). And/or, I would expect an "accountable Poll"; each Voter must post with some kind of explanation of their Vote, and any who failed to account for their Vote would have their Vote removed.

A Serious(tm) Poll, basically. Then 'maybe'.

 

[Shar Dundred]

I see the point of this, I really do, but you cannot make it that simple.

Let's say someone is appointed as judge.
If said judge is now working on his judging and something
happened, i. e. a close relative died, will you blame him for
needing some time first?
Or even punish him for it?

Again, I understand that it might be frustrating, but it isn't
that easy to generalize these things.

 

[Murlocologist]

I see the point of this, I really do, but you cannot make it that simple.

Let's say someone is appointed as judge.
If said judge is now working on his judging and something
happened, i. e. a close relative died, will you blame him for
needing some time first?
Or even punish him for it?

Again, I understand that it might be frustrating, but it isn't
that easy to generalize these things.

Yes but these are extreme cases. Of course, if judge just disappeared, haven't logged in or posted anything for a month, you can't -rep him until you see if he really had any reason other than laziness to abandon contest. But, if you see him posting around the site, logging in and out here and there, than (if he had any reason that held him from doing task) he could also notify host he wants to step back from being a judge.

 

[Shar Dundred]

I don't disagree with you, just saying it's hard to make a general rule for this.

 

[deepstrasz]

Of course, if judge just disappeared, haven't logged in or posted anything for a month, you can't -rep him until you see if he really had any reason other than laziness to abandon contest.

What if they lie and they've actually been careless all that time doing some other unimportant stuff?

 

[The_Silent]

Will scare potential judges away.

 

[Shar Dundred]

That too.

 

[Murlocologist]

Well for lazy potential judges responsibility surely will be scary.

 

[Kyrbi0]

What if they lie and they've actually been careless all that time doing some other unimportant stuff?

We can't control human nature. Everything here is built on the assumption of integrity. Whatever safeguards we can erect, we have (i.e. "post a WIP/large image" to prove you made it yourself post-Contest start, etc).

I see the point of this, I really do, but you cannot make it that simple.

Let's say someone is appointed as judge.
If said judge is now working on his judging and something
happened, i. e. a close relative died, will you blame him for
needing some time first?
Or even punish him for it?

Again, I understand that it might be frustrating, but it isn't
that easy to generalize these things.

I don't think it's that hard to generalize.

If a Judge has an absence greater than X days/weeks with no communication/notification, his Judging will be passed on to another (and punishment maybe?). If a Judge has an absence greater than Y days/weeks but does so for a legitimate reason (i.e. illnes/injury/life upheaval/etc) (determined by Contest, or by Staff), extenuating circumstances will be taken into account and -----(whatever happens)----"

 

[KILLCIDE]

If failing to judge entries has punishments, then being a judge in the first place will have to be more rewarding.

 

[Kyrbi0]

Well we have started giving Judges rep for Judging (long time coming I think). More?

 

[KILLCIDE]

Not all contests reward judges with reputation, and there are also a handful of people who don't even care about reputation. I don't have any better suggestions, but I was just saying what I thought should happen.

 

[Kyrbi0]

Not all contests reward judges with reputation

Yeah, it's kinda new. I think it's a great idea, though.

...and there are also a handful of people who don't even care about reputation.

That's true, but there's really nothing else that can be given. :<

I don't have any better suggestions, but I was just saying what I thought should happen.

That's cool. I wasn't attacking your idea, just continuing the conversation.

 

[KILLCIDE]

Well, there are still the "custom flag" and "user rank icons" that Archian has talked about in the past. If that is still on the table, I would suggest something like "if a user gets XX approvals from a Host for succesfully judging a contest, they get the 'judge' flag or the 'judge icon' blah blah"

 

[Kyrbi0]

Hm, ok.

 

[Murlocologist]

Well, there are still the "custom flag" and "user rank icons" that Archian has talked about in the past. If that is still on the table, I would suggest something like "if a user gets XX approvals from a Host for succesfully judging a contest, they get the 'judge' flag or the 'judge icon' blah blah"

There is a rule that says it is not advisable for same person to be judge all the time. I wouldn't still go so far and overvalue judges tasks to award them with what I find ridiculous and community finds as ''special awards''. There are many judges reviews in the past of contests that were rubbish and I would disagree with rewarding such judges either way. Since tagging a review as trash is quite subjective matter, you could hardly reach a consensus.

 

[KILLCIDE]

There are many judges reviews in the past of contests that were rubbish and I would disagree with rewarding such judges either way.

That's what I said "if a user gets XX approvals from a Host for succesfuly judging a contest."

 

[Murlocologist]

That's what I said "if a user gets XX approvals from a Host for succesfuly judging a contest."

Well, from how it seems from third person view, host never blamed judge for delivering a bad reviews. And without doubt there has been contests with bad reviews. The problem there seems to be that hosts think they would step out from kindness zone if they would criticize how judge approached his task.

 

[Chaosy]

I see the point of this, I really do, but you cannot make it that simple.

Let's say someone is appointed as judge.
If said judge is now working on his judging and something
happened, i. e. a close relative died, will you blame him for
needing some time first?
Or even punish him for it?

Again, I understand that it might be frustrating, but it isn't
that easy to generalize these things.

I've only lost one close to me, and that was my dog.
And as far as most pet owners are concerned, those are just as important as any family member sentimental wise.

I went to school as normal the day after.
I went and sobbed for half an hour during one of the lessons but otherwise I did considerably fine.

So taking a long break makes no sense to me.
And even if that would be needed - somehow. Most people would be accepting of it, if they bother to tell us in advance.
Not "My aunt died last week so I was unable to work for x days." but rather "My aunt died today, I wont work for the next x days."

This would probably fix this as well:

What if they lie and they've actually been careless all that time doing some other unimportant stuff?

 

[KILLCIDE]

I went to school as normal the day after.
I went and sobbed for half an hour during one of the lessons but otherwise I did considerably fine.

How someone deals with loss is extremely subjective.

 

[Chaosy]

True enough. But I think it's minority that become incapable of acting out of sorrow.

I am not entirely sure, but I don't think you can skip work because of mourning. (other than going to the funeral if that's scheduled during work hours)
So, if everyone was like that there would be a lot of people getting kicked.

 

[Murlocologist]

It is not important what is the occurrence that will keep them away from doing task, the only relevant thing contest-wise is that they notify host they won't be able to do judging and ask to be replaced. Their reasoning does not matter either way.

 

[KILLCIDE]

I'm under the assumption that you made this thread because of the Zephyr Contest. I should let you know that IcemanBo is very well informed about what is going on between the 3 judges. The code judge has a lot of code to go through, and all though time has past, it seems wasteful to replace him, thus making the work he has already done so far useless.

 

[Murlocologist]

I'm under the assumption that you made this thread because of the Zephyr Contest. I should let you know that IcemanBo is very well informed about what is going on between the 3 judges. The code judge has a lot of code to go through, and all though time has past, it seems wasteful to replace him, thus making the work he has already done so far useless.

Your assumption is correct. But the only link between this thread and that specific contest is your above statement.

I find the 1 month after poll end enough time for judge to deliver judging. And how IcemanBo is dealing with his judges is relevant to only four of them. The fact relevant for me is that the mentioned contest is now 6 months long. And I already expressed my opinions in that thread regarding the situation.

 

[KILLCIDE]

Right. A judge should be replaced after a certain time of inactivity, but the only reason the code judge wasn't replaced was because it would be shameful to replace him after the amount of work he has already done.

It's already hard enough to find a competent judge for the Zephyr Contest, so finding a replacement judge would just be... meh. I don't really pay attention to other contests, but do they experience the same issue as the Zephyr in terms of the amount of time it takes to post the results from the end of the poll?

 

[Murlocologist]

I don't really pay attention to other contests, but do they experience the same issue as the Zephyr in terms of the amount of time it takes to post the results from the end of the poll?

Frequently. At least that's how I see it, slow motion between the end of poll to the posting of results thread.

 

[Kyrbi0]

I don't really pay attention to other contests, but do they experience the same issue as the Zephyr in terms of the amount of time it takes to post the results from the end of the poll?

I have taken part in a Contest that took almost 5 months from Poll to Judging... But it was a big one (Wc3C 3rd Hero Contest).

I'd say it's reasonably frequent. I don't think any of the Contests I ran took more than a few months to have things finished. Some are faster than others.

 

[deepstrasz]

If failing to judge entries has punishments, then being a judge in the first place will have to be more rewarding.

What about contestants? Why shouldn't they be negarepped if they fail to put their final entry or even a WIP before the contest ends although they wrote they'd participate. I don't like this discrimination of judges.

So taking a long break makes no sense to me.

Please spare us the bullshark. Animals are not relatives. It's absolutely not the same thing. But we aren't just debating loss of close persons. It's about everything than can happen: depression, accidents, illnesses etc.

Not "My aunt died last week so I was unable to work for x days." but rather "My aunt died today, I wont work for the next x days."

It doesn't always work that way. People are different. They deal with matters differently. They can just become careless because of a tragedy like that and think not to tell anyone until they are ready to.

 

[Chaosy]

Please spare us the bullshark. Animals are not relatives.

Objectively, no.
Sentimentally, yes.

A person can value a video game more than one's family. (Google can give you many examples of this.)
So I don't see why it's so weird to value a dog that might have been around since you were born, just as your parents.

But we aren't just debating loss of close persons. It's about everything than can happen: depression, accidents, illnesses etc.

Not sure what to say about depression, but accidents and illnesses should be called in before you take a week's 'leave'. Just like you'd call in sick for work the same morning, not the week after.

It doesn't always work that way. People are different. They deal with matters differently. They can just become careless because of a tragedy like that and think not to tell anyone until they are ready to.

This is bullshit.
If I break my arm, I cannot bother to write a text message which takes 2 minutes? and this can be done on a phone, so there is no excuse like "I have no computer in the hospital"
Even in case of depression, sending a text message takes no emotion. I am no master psychologist, but it's very much easier to express oneself through the internet, compared to saying something face to face.

edit:

What about contestants? Why shouldn't they be negarepped if they fail to put their final entry or even a WIP before the contest ends although they wrote they'd participate. I don't like this discrimination of judges.

Interesting point.
Though the difference is that, a contestant leaving does not impact others.

 

[deepstrasz]

Though the difference is that, a contestant leaving does not impact others.

It does when there are few contestants and only three places to be awarded. It would look weird with three contestants if the fourth one suddenly disappears.

This is bullshit.

Not at all. As you've written:

A person can value a video game more than one's family. (Google can give you many examples of this.)

People are weird different so, unexpected things like not giving a duck to leave a message on a forum might just become a realization.

 

[Murlocologist]

What about contestants? Why shouldn't they be negarepped if they fail to put their final entry or even a WIP before the contest ends although they wrote they'd participate. I don't like this discrimination of judges.

How is it same? Contest is usually opened for 1-2 months, after deadline is locked and goes to poll. Everything that happened during opened contest matters to no one, what matters is what is left after locking it. You make absolutely no commitment to anyone by posting wips. Discrimination of judges? Please...

 

[deepstrasz]

Everything that happened during opened contest matters to no one, what matters is what is left after locking it.

Oh really? So, it's decent to cancel a contest due to a lack of a proper number of contestants (that might just drop out for no reason) but it's indecent for judges to disappear but decent to negrep them after doing so.

 

[Murlocologist]

Oh really? So, it's decent to cancel a contest due to a lack of a proper number of contestants (that might just drop out for no reason) but it's indecent for judges to disappear but decent to negrep them after doing so.

Basically, yes.

 

[deepstrasz]

Basically, yes.

Good luck finding a decent number of judges after detracting reputation points from former judges when most people are irresponsible and this site is for hobbyists (don't forget most people here are kids).

 

[Murlocologist]

Good luck finding a decent number of judges after detracting reputation points from former judges when most people are irresponsible and this site is for hobbyists (don't forget most people here are kids).

I'd rather rely on public vote if I had to chose between poll and irresponsible judges.

 

[deepstrasz]

I'd rather rely on public vote if I had to chose between poll and irresponsible judges.

Leaving it only by poll (normal users) would lead to what happens with the resource sections when bad stuff gets 5/5 and good stuff 1/5 on a whim.

 

[Kyrbi0]

Leaving it only by poll (normal users) would lead to what happens with the resource sections when bad stuff gets 5/5 and good stuff 1/5 on a whim.

Well, that's only if we stick with Polls as we know them now.

Enforcing either (A) Voter Accountability (i.e. through requiring a post 'explaining' ones' vote), or (B) Improved Polling (i.e. through a preferential voting method, such as the Modified Borda Count or Schulze), or both, will almost certainly mitigate that issue greatly.

 

[deepstrasz]

Enforcing either (A) Voter Accountability (i.e. through requiring a post 'explaining' ones' vote), or (B) Improved Polling (i.e. through a preferential voting method, such as the Modified Borda Count or Schulze), or both, will almost certainly mitigate that issue greatly.

A-requires somebody to actually check the explanations and decide if they are what they're supposed to be (hence basically judges?)
B-from what I understand, every user has to give points to all candidates descending or ascending (doesn't really matter) and finally only one of them will have the most points (isn't that basically the same as a one vote poll?)

Anyways, the more complex it gets, the less voters we might get. So, I'm not entirely sure what to do especially if we at some point have no one to judge.

 

[Deleted member 219079]

Judges are fellows who happen to have some level of expertise on the judged field they've been appointed to. I wouldn't expect them to exceed the average hiver on any other property, including maturity regarding scheduling and self-evaluation.

Before resorting into system changes it'd be better to try some rule changes, I'd like removing "you have not judged many other hive workshop contests recently". This is so that publicly acclaimed judge wouldn't be turned down just because of judging previous one. Of course there might have been deeper reasoning behind that rule, but the community is so small that I don't like judge selection being that picky.


And please no more user promoting, mod rating removal was enough cancer for this year.

 

[Murlocologist]

Before resorting into system changes it'd be better to try some rule changes, I'd like removing "you have not judged many other hive workshop contests recently". This is so that publicly acclaimed judge wouldn't be turned down just because of judging previous one. Of course there might have been deeper reasoning behind that rule, but the community is so small that I don't like judge selection being that picky.

That rule is ignored anyway, so leaving it or removing it would make no difference to state of judging.

 

[Deleted member 219079]

How is it ignored?

Even if it would be ignored to some extent, I don't promote system changes if not necessary as they would result in submission ranking not being consistent over time. Besides formal removal of that rule would reduce the threshold of previous judge being re-elected, on behalf of both parties (staff wouldn't hesitate as much to ask for judging the next one and previous judge wouldn't hesitate as much to ask for it).

 

[Murlocologist]

How is it ignored?

Even if it would be ignored to some extent, I don't promote system changes if not necessary as they would result in submission ranking not being consistent over time. Besides formal removal of that rule would reduce the threshold of previous judge being re-elected, on behalf of both parties (staff wouldn't hesitate as much to ask for judging the next one and previous judge wouldn't hesitate as much to ask for it).

That rule is just an outdated information who nobody finds to be relevant in process of arranging contests anymore. It is making no limitations at all, same persons can be, and some are, judges on more less constant basis.

 

[Deleted member 219079]

That rule is just an outdated information who nobody finds to be relevant in process of arranging contests anymore. It is making no limitations at all, same persons can be, and some are, judges on more less constant basis.

I can run over a baby with my car while not permitted in rules.

 

[Kyrbi0]

I think the point he's making is that that rule is essentially non-existent; Judges are chosen & re-chosen again & again. No one notices & no one penalizes for it; in all senses it is 'just writing'.

Personally I can corroborate that; I didn't even know that was a rule.

 

[Deleted member 219079]

Then ask for staff to remove it.

I think the point I'm making is that there's no need for system changes, rather get two mods and if necessary remove one which doesn't make it in time.

 

[GhostThruster]

speaking as a former judge of contests, i agree that in many circumstances a judge can and should be dropped or replaced.

i don't agree with a neg rep as punishment. that's reserved for rule-breaking. it's just stupid for punishing people that were unable to complete something that is a voluntary, hobbyist activity.