SRL Archive - News and General

A couple of months ago I did a thread on JM competition #3 - I have not forgotten this, just haven't had the time to properly start up that project as I've had other projects on my hands. (I like having projects :p)

Now here's my question + context:
Would you guys, SRL Members, accept being judged by other SRL Members?

I personally do not have the time to judge scripts - and few devs do. What was done for the JM competition was a team of judges, which allowed the judging to go very fast.

If you guys accept being judged by those the same rank as you (although people will still be handpicked from applications) then competitions for SRL Members could be re-started.

What are your general views on this?

~RM

Why not have SSRL members judge? That's assuming you get enough to do the judging.

From: Coh3n
Why not have SSRL members judge? That's assuming you get enough to do the judging.

Key part of sentence ;)

~RM

You sort of need one person judging all scripts, to set a standard. One person could believe a script is amazing, while another could think it is rubbish. I don't think it would be as fair :(

From: Rasta Magician
Key part of sentence ;)

~RM

Yeahh…

I'm up for judging again, it was fun. I actually think still I have my marking template laying around somewhere. But with the other projects we have going on, I can't see me having the time.

From: Ejjman1
You sort of need one person judging all scripts, to set a standard. One person could believe a script is amazing, while another could think it is rubbish. I don't think it would be as fair :(

A big enough team evens out such things. By having several people judging all the scripts, you can just average the scores so it returns a fair score. Last competition went quite fairly, I believe :)

~RM

From: Rasta Magician
A big enough team evens out such things. By having several people judging all the scripts, you can just average the scores so it returns a fair score. Last competition went quite fairly, I believe :)

~RM

Ok, I was under the impression that you'd have, 8 people just 8 scripts, once. If each judges and then average, I see how it would work. Thanks for clarification.

I can see flaws in this, which I can't really explain well, but I'll insert a scenario to try and make sense of it.

Contestant A makes a script which has very advanced techniques. Judges A and B have not seen this technique used before, and give the contestant a high mark. Judge C has seen it, however, and gives a low mark because the contestant used it improperly. This would average out to a high mark overall though.

Does that make sense? I have been away from scripting for so long that I don't know if it would happen within the SRL members or not. Who knows, I may be completely off track.

I personally wouldn't enter a scripting competition, but if I were to enter, I would prefer being judged by people with more knowledge than myself.

From: 3Garrett3
I can see flaws in this, which I can't really explain well, but I'll insert a scenario to try and make sense of it.

Contestant A makes a script which has very advanced techniques. Judges A and B have not seen this technique used before, and give the contestant a high mark. Judge C has seen it, however, and gives a low mark because the contestant used it improperly. This would average out to a high mark overall though.

Does that make sense? I have been away from scripting for so long that I don't know if it would happen within the SRL members or not. Who knows, I may be completely off track.

I personally wouldn't enter a scripting competition, but if I were to enter, I would prefer being judged by people with more knowledge than myself.

Last competition, the way this was managed was this:

A thread with the script is posted.

All judges post their comments without showing the assigned marks. Once all the judges have read each others posts, (if they missed anything), they can adjust their marks accordingly.

Which means:
Judge A and Judge B gave 100% for creativity and technique, while Judge gave 90% for creativity and 70% for tecnique.

A and B read C's comments, and realize that this technique had been used before and that it was wrongly used, therefore they can adjust their marks accordingly.

Also, only a bad judge would judge something he doesn't understand. Therefore I imagine the judges would go through the trouble of properly understanding the person's script.


But thank you for your reply on that last point.

~RM

From: Rasta Magician
Last competition, the way this was managed was this:

A thread with the script is posted.

All judges post their comments without showing the assigned marks. Once all the judges have read each others posts, (if they missed anything), they can adjust their marks accordingly.

Which means:
Judge A and Judge B gave 100% for creativity and technique, while Judge gave 90% for creativity and 70% for tecnique.

A and B read C's comments, and realize that this technique had been used before and that it was wrongly used, therefore they can adjust their marks accordingly.

Also, only a bad judge would judge something he doesn't understand. Therefore I imagine the judges would go through the trouble of properly understanding the person's script.


But thank you for your reply on that last point.

~RM

Ahh, I see. That makes much more sense than what it worked out to in my head.

I didn't want to seem negative, if it came out that way. I like the idea, just to let you know. I wont prematurely promise to be able to help with anything until I can determine what my work flow will be like from now on.

If anything, I'd prefer to be judged by SRL Members. It's more of a communal feedback, helping-out sort of thing, I think.

Don't care.

From: Quickmarch
Don't care.

Alright man thanks for coming out. :)

I think its a bad idea. If you are going to have multiple judges , have multiple (the same) judges critique each script, and then choose their median score as the final score for the script.

Otherwise the competition wouldn't be fair.

From: lordsaturn
I think its a bad idea. If you are going to have multiple judges , have multiple (the same) judges critique each script, and then choose their median score as the final score for the script.

Otherwise the competition wouldn't be fair.

Isn't that what I said…?

By having several people judging all the scripts, you can just average the scores so it returns a fair score

~RM

Good Idea!

But why not let everybody judge, by means of making the scripts public? And give everybody a vote? Each Script Thread might include a predefined poll with fixed parameters for each script. Usability, profitabilite, stability, readability, etc. I guess that would be easiest way. SRL Members votes could be giiven a double count priviledge, so more power in the vote. I think that is only fair. Devs maybe three, dunno bout that.

Anyway, I think it is a good time to start develop such a contest, but take your time. Holidays are coming and that will raise activity on the board. Hopefully we can than open up again, pfrt.

From: WT-Fakawi
Good Idea!

But why not let everybody judge, by means of making the scripts public? And give everybody a vote? Each Script Thread might include a predefined poll with fixed parameters for each script. Usability, profitabilite, stability, readability, etc. I guess that would be easiest way. SRL Members votes could be giiven a double count priviledge, so more power in the vote. I think that is only fair. Devs maybe three, dunno bout that.

Anyway, I think it is a good time to start develop such a contest, but take your time. Holidays are coming and that will raise activity on the board. Hopefully we can than open up again, pfrt.

we can then* open up again. :P

Based off reputation. If I see a script made my Fawk over SRL Member, then I choose Fawks (if they same script). Therefore, you will have people crowd around the cooler person, persay, and it'll be a popularity contest.

Popularity comes from talent here ;)

One of the older SRL member competitions (In wich only 2 people entered) isn't even done being judged.

People hear this stuff, get excited to script/vote/judge, then we end up getting a 5 or less people turnout, and on top of that the judges become busy/lazy and nothing even happens. Let's AT LEAST get some people to friken judge the unjudged competition first before starting this! :D

From: YoHoJo
One of the older SRL member competitions (In wich only 2 people entered) isn't even done being judged.

People hear this stuff, get excited to script/vote/judge, then we end up getting a 5 or less people turnout, and on top of that the judges become busy/lazy and nothing even happens. Let's AT LEAST get some people to friken judge the unjudged competition first before starting this! :D

Because the people who set up the competition are the ones who know the criteria by which to mark, and the participants are inactive :rolleyes:

@Fawk: I'm not quite sure that would indeed be fair :s I'll post my reasoning later, I'll be on the way to my exam now :)

~RM

Good luck RM!

I liked how the markings in JM Comp #1 was.. What was wrong with that, WT?

From: YoHoJo
One of the older SRL member competitions (In wich only 2 people entered) isn't even done being judged.

People hear this stuff, get excited to script/vote/judge, then we end up getting a 5 or less people turnout, and on top of that the judges become busy/lazy and nothing even happens. Let's AT LEAST get some people to friken judge the unjudged competition first before starting this! :D

Perhaps it's because we can't actually see the forums for judging..

Yes but I don't think that many members are really capabale of judging. Find the few and it's fine.

From: Nava2
Perhaps it's because we can't actually see the forums for judging..

Well, fawk most definitely can and a copy of everything was posted in ssrl, so at the time, so could you ;)

~RM

It wouldn't bother me in the slightest being judged by other members.

To be honest, I supported this idea once, but i think it's not right anymore due to people taking there friends sides?
not everyone would do it but there may be some people out there who would :/.

example:
Refs favor Lakers(Kobe Bryant).
Belive it or not <3 ^

so i would personally like devs to judge! OR maybe SSRL :|

From: ~~Joker~~
To be honest, I supported this idea once, but i think it's not right anymore due to people taking there friends sides?
not everyone would do it but there may be some people out there who would :/.

example:
Refs favor Lakers(Kobe Bryant).
Belive it or not <3 ^

so i would personally like devs to judge! OR maybe SSRL :|

They wont know who posted what script unless they know the script on beforehand, and when more than one person is judging, their vote / marks wont count as much as it would if only one person were judging :)

You would have to pick honest people to do the judging seeing how they could see if a friend entered or something, but other than that who really cares? Even keeping the judges anonymous would keep everyone from complaining too

From: Zyt3x
They wont know who posted what script unless they know the script on beforehand, and when more than one person is judging, their vote / marks wont count as much as it would if only one person were judging :)

Ah! i see :) well than it sounds like a good plan! :garfield:

Seems like the idea in general is a go-go. Most problems that have been pointed out can be overcome easily and fairly enough.

Watch for future threads.

~RM