Vi-Poi
Administrator
Premier of Earth
PL: 434,410
Soul(x40P), Overdrive(x43)
Zeni: 1,247
Tag: @vipoi
Posts: 2,833
|
Post by Vi-Poi on Sept 29, 2015 22:23:06 GMT
Hey everyone! As many players may notice, the T2G thread has grown heavy and unwieldy, with wait times at our highest levels ever. We need more volunteers, but we're quickly reaching the point where the number of staff we have is being offset by the rate in which they get disinterested in pushing the cogs. As anyone who's done a lot of it will tell you, grading is a monotonous, time-consuming, and often thankless job. In working with some programmers over this past month who develop plugins for proboards, I've been able to acquire a trio of plugins that with some careful use can mimic exactly the process of our grading formula. These plugins issue PL based on WC, quality, and items automatically as soon as the user clicks the Create Thread or Reply buttons. These plugins are dynamic enough to work perfectly under many conditions, but there are some areas where they do not work. 1) They cannot account for WIPs, changing rewards based on edits, or mobile phone posting. 2) They cannot account for individual post quality awards or deductions. 3) They work only for PL, and not for Zeni. Several schemes to address these faults are operating in my test site. For individual post quality awards, hidden groups can distribute a 'PL wage' to the player when the player is added to it, and this wage can also be negative. To avoid someone's PL dropping during this distribution, their overall quality scores can be slightly low-balled, so the wage allotments are always in positive territory. Further, more frequent quality assessments can diminish this problem. Not accounting for WIPs can be handled by placing your WIPs in your mail, and mailing it to yourself. When it's ready to post, you can pull all the material into the thread at once. As always, Word Processors are encouraged. Posts that have been done on mobile can be graded the old fashioned way. As for not earning Zeni automatically, this is the big problem in my mind. Users wouldn't be able to request PL in a thread without having their gained PL deducted and their Zeni done manually. The plugins do provide for the ability to only operate in certain boards however, so with the creation of a Zeni board for zeni earnings, this issue can be sidestepped altogether. Automated security measures, such as protecting against deleting and reposting and those to prevent someone trying to fool automation by plagiarizing material from this site or from Google are already being successfully tested. Those are the drawbacks, though the benefits would be substantial. There would be absolutely no lag time between PL threads and PL awards. Right now this is all on my test site, which has a lot of new features that may or may not ever become a part of Souls. This is still far off beyond the horizon, but before I show the demo to staff and have them play around with it I wanted to get the general feeling for automated grading here. Note that this wouldn't change the formula at all, whatsoever, nor change the fact that your material is reviewed -- it'd just free the graders to be readers and reviewers rather than button mashers. Also keep in mind that while the situation of T2G might get better, it might very well get worse as our numbers increase. It is entirely possible that we'll be crushed under our own weight, if there isn't a change. Please let me know what you think about a possible change, and thank you for the input. Zucceta President Bao Pieter Wolfbane Reikiko Kaula Bing Gan Nicolas Mclendon Hyoza
|
|
|
Post by Raisu Hanamura on Sept 29, 2015 22:38:15 GMT
As tempted as I am to make a robot overlord joke, I have a few serious thoughts on the matter.
The potential flaws have been already outlined. I can't think of anything else to add to those. But I think they're big enough to most definitely work against the system.
I, personally, post from mobile a lot. I'm doing it now, in fact. But I prefer to post from a computer. It is not uncommon for my posts, especially in longer threads, to be a mix of mobile and computer posts. I feel as if this would make things difficult. I assume you've devised a way to track which posts have been already graded by the automated system and the like, but I imagine it would make things difficult when half the posts are mobile and half are not.
Also, I am curious if it can differentiate between various items in inventories. I have seen people with two different sets of weights. Or if it can recognize gravity threads.
I, mean, I personally like the idea of it. I just wonder if it'll create more problems.
|
|
Vi-Poi
Administrator
Premier of Earth
PL: 434,410
Soul(x40P), Overdrive(x43)
Zeni: 1,247
Tag: @vipoi
Posts: 2,833
|
Post by Vi-Poi on Sept 29, 2015 22:40:51 GMT
Those are legitimate points. Right now the mobile posting is being worked on, so there is no definite answer there besides it'll have to be done manually for now. Part of why I posted this is to gauge just how many mobile users we have.
It can easily account for all different items, gravity, and every other factor in PL grading.
|
|
|
Post by Letrune on Sept 29, 2015 22:54:18 GMT
I'd suggest that at least a mod should verify the program's end result, and the gains should only happen after the thread is closed.
|
|
|
Post by Aiami Lin on Sept 29, 2015 22:58:07 GMT
Overall I like the idea. I don't know a whole lot about the grading process, but from what little I do understand, it's not complex, just tedious. That makes it perfect for a program like you described, Vi Poi.
Raisu already posted some of the things I was wondering about, which would be different variables in a thread. As well, would this be primarily used for solos, or can this be used for multi-person threads? Just to establish that.
Finally, I'm glad a review process will remain. Should help fight things like bloating, as well I would like to hope I get better at writing as I continue to do so, and like to see that reflected in the grading.
I am definitely for this idea. Hopefully it will allow staff and any volunteers more time to enjoy the site and RP than be bogged down with grading.
|
|
|
Post by Pieter Wolfbane on Sept 29, 2015 23:44:51 GMT
Would it be possible to actually create a sequence of codes to:
1) Identify player requests for either Zeni or Power Level at the beginning or end of the thread? Have a specific plugin that asks, "Power Level or Zeni," which can be marked and have said option be changeable later?
2) Identify the items as different codes that register the purchases of items?
|
|
|
Post by Kizuki on Sept 30, 2015 0:07:42 GMT
Well, we won't know until we try, right? Perhaps take some old, already graded posts for PL and do some test runs. I'm all for any sort of ease of grading, be it faster for us or an easier time for the mod's.
Am I understanding it correctly that each "player/character" is going to have their quality multiplier input in this system? If not, I don't really understand that portion of it .. lol. I guess it would be easier to understand once a few test runs are done though, no?
|
|
Zod
Newcomer
Tag: @zod
Posts: 44
|
Post by Zod on Sept 30, 2015 0:51:19 GMT
I know I don't have a character yet but I must push something... there is no automated system that can grade quality. What if someone's quality goes up? What if they become busy and it falters? How do you identify what quality is at which grade? This is actually part of why I mentioned publicly posting up the formula used. I do see an easy solution to this problem, however.
As I've mentioned, I've been part of many boards that have lasted for even over a decade. A big part of what kept them together was a three administration system. The bon-a-fide owner, personal relations, and a workhorse. Basically, from what I've seen, you guys are missing your workhorse.
Generally boards like this one use division of labor so that one member of staff ONLY does upgrades, and what he says the upgrade is cannot be questioned. On top of this, that staffer can also add or remove staff for grading as he sees fit; I'd like to add that people who don't do well communicating with others are great for this job, since they no longer need to juggle helping out with questions, approving profiles, and taking care of new maneuvers where everything becomes a debate. This is the workhorse... he does his job and is rewarded with the freedom to be an asshat to everyone who slows him down x.x; (I can remember so many I despised yet couldn't live without...)
You then take a second administrator with good personal skills and have them deal with character profiles and the like. The people that he has gained an innate trust with can then be the mods that help him deal with questions, suggestions, character profiles, maneuvers, etc.
The 3rd administrator is well... generally the owner and disciplinary, while also being an all-rounder.
With this, you don't have to worry about the board coming to a hault because one member falls behind and things will continue even if a single suggestion starts taking up 3 or 4 staffers at once. Of course, this does get in the way of the democratic solution for a more corporate one, but you'll be able to continuously run the board with no problem.
|
|
Vi-Poi
Administrator
Premier of Earth
PL: 434,410
Soul(x40P), Overdrive(x43)
Zeni: 1,247
Tag: @vipoi
Posts: 2,833
|
Post by Vi-Poi on Sept 30, 2015 1:26:41 GMT
Letrune, that possibility is being looked at, but it looks unlikely unless we can tie it to some sort of proboards command like Lock Thread. Aiami Lin, it works for both groups and individuals. Kizuki, it's still being tested, and we won't know for sure until there are a lot of people operating the system in tandem whether or not it's working correctly. That's for the scale up tests I am planning with the help of others. Zod, we don't want to operate with divisions of purpose for a lot of reasons -- giving one person an implementation crown could lead to a lot of mistakes being made. Every professional game is made with a development team for this very reason, and making one person to be the 'workhorse' is unfair and frankly impossible over time, given the scope of the grading work that needs doing. There is an automated system that can grade quality (the one I'm testing ) because of the deliberate, inertial, and consensus-based way our quality judgments are made and effected.
|
|
Zod
Newcomer
Tag: @zod
Posts: 44
|
Post by Zod on Sept 30, 2015 1:52:48 GMT
tbh, this board isn't that active compared to the ones I was on before. Not only that but those boards had no word limits or requirements. The workhorse is simply someone who doesn't have to deal with "newbs" and only has to grade. what they have to do, need to do, and how to regulate it becomes much more clear when division of labor is added into it. As for professional games needing development teams, that's why I stated that the administrators could make temp or perm mods depending on how busy it is at the time... those people would be your "team." If you wish to learn more about this, direct yourself to Druker's Management and the segment about dividing out jobs into what the management team specializes in. As for "the crown," this is why 3 administrators are necessary to make the the underlying teams. If one administrator gets out of hand, the other two pull back on the reigns. I believe I stated, this is what boards that lasted over 10 years did to cope with an influx of member activity. During busy times, they were receiving over 10 regular threads along with personal sagas to grade (personal sagas are character development threads that can be over 40k words long). The threads were graded by the work horse admin along with the mods that only graded material. The new members were being welcomed and helped along by the friendly admin and his staffers. The owner then just sat back and picked up the slack and disciplined members and staff that got out of line. I should also note, the workhorse admin, though hated for his personality, is also generally the most respected member of the staff x.O; I remember how Pozest Star always did all her work on sunday for almost 8 hours when she was taking 6 college courses during the week, ha ha. She quickly tossed up the middle finger to anyone who tried to get her to grade something on any other day and let everyone know "I'll get to you sunday and sunday only" She was a laugh riot, fun to tease, but a little introverted because of being bullied in highschool. She was the most respected member of staff, even more than the board owner who had to deal with twins and his wife losing her job x.x; she just did that much for us
|
|
|
Post by Kizuki on Sept 30, 2015 13:35:09 GMT
Letrune, that possibility is being looked at, but it looks unlikely unless we can tie it to some sort of proboards command like Lock Thread. Aiami Lin, it works for both groups and individuals. Kizuki, it's still being tested, and we won't know for sure until there are a lot of people operating the system in tandem whether or not it's working correctly. That's for the scale up tests I am planning with the help of others. Zod, we don't want to operate with divisions of purpose for a lot of reasons -- giving one person an implementation crown could lead to a lot of mistakes being made. Every professional game is made with a development team for this very reason, and making one person to be the 'workhorse' is unfair and frankly impossible over time, given the scope of the grading work that needs doing. There is an automated system that can grade quality (the one I'm testing ) because of the deliberate, inertial, and consensus-based way our quality judgments are made and effected. Well, I will say that I am REALLY excited for that. I'm sure there will be bumps along the way of getting this to work, but it'll push things along further. Being able to at least KNOW what you're overall gains would cause a big boost in member activity, in my opinion, even if we would have to wait for an admin to actually add them to our character. Amazing what technology can do now, like being able to set parameters as to what to look for with quality.
ANYWAY -- If there is another guinea pig needed to test this process feel free to throw me in the mix.
|
|
|
Post by Natto on Sept 30, 2015 13:37:25 GMT
I think if that work fine, that's an amazing thing to be implemented on WoDs! It would make everything more dynamic and take a heavy load out of the graders backs! =) Good job Vi-Poi, I wish it becomes a reality soon!
|
|
|
Post by Letrune on Sept 30, 2015 13:39:34 GMT
Just an idea, it might run already... The automata gives the normal rate, the bonus comes by a mod's aggreement?
|
|
|
Post by Hyoza on Sept 30, 2015 14:34:51 GMT
What people need to understand in regard to an automated grading system is this:
The site struggles under the weight of its own expansion. Many new members may join, but the number of suitable staff candidates will not necessarily grow to match. There've been serious failures in the past with regards to staff recruitment and understandably standards are much higher now for new staff candidates. Every new member that is active past their beginner's post will generate at least a small addition to the grading/approvals workload for staff. The disparity between staff recruitment and new member sign-up is visibly leading us down a road where the staff become totally incapable of keeping up with the workload.
I wouldn't personally advocate making the requirements for new staff easier to meet, especially in light of the recent additions to their role regarding purely mechanical site design. The site needs its staff to be on the ball and playing fair, and right now we're handling that aspect of things pretty well.
It seems to me, then, that the ONLY real solution worth considering is automation of grading. It is by far the largest workload for staff and volunteers, and even though Vi-poi's proposed system has flaws - the worst that can happen is a few posts and threads end up needing to be graded manually. It will still remove the vast majority of the grading load from staff, and give them more time to focus on their other duties, including:
- Control of Afterlife NPCs - Technique Applications - Mechanics design and discussion of new suggestions by the members - Character Applications
And so on.
I'm for it, it may not be an easy transition but this needs to happen - right now the site is expanding too fast for its design to cope. Automated grading is at least a partial fix to this, and I'm in support of its use.
|
|
Ryuga
Newcomer
PL: 17,121; Hidden Potential (x4); Items: One-Use Spacepod; Zeni: 750
Tag: @ryuga
Posts: 44
|
Post by Ryuga on Sept 30, 2015 17:56:54 GMT
I like the idea of automated grading if it can be handled correctly, and obviously you are testing it out in length long before even bringing it here for a trial, so I have faith that it'll work well. The only thing I am curious about is one of Pieters points actually, how to let the program know that you want Zeni for your thread instead of PL? I know you said there was no set system for Zeni yet but is there a way for the program to tell the difference? In regards to Zeni grading, I have a monetary system on my site that can be altered 7 ways from Sunday to do what you want it too. It's just called the monetary system: www.proboards.com/library/plugins/item/90I've found it extremely helpful and I dont doubt that the more you play with it, it could work well for Zeni here as well. -Tomo
|
|