Userscripts.org

Does anyone know of any good references for working point systems. Right now users who have been here for 2 years or make scripts with tons of users get the same voting rights as a spammer who just signed up.

I've read that news.ycombinator.com doesn't give voting rights to members until they earn enough points. And new members are voted down if they don't conform with the norms of the site. (so if there is voting on scripts you could imagine someone posting a scrap all script would get voted down :) )

I think also adding transparency to voting will make it easier to spot the spammers.

Stackoverflow has an interesting concept for what reputation enables you to do: http://stackoverflow.com/faq

Maybe an altered model of that kind of reputation system should be brought here.
Now, question arises, what does make a person reputable than a good set of scripts?

I can remember that in the middle on '90 I traveled to the US, on the plane we received a form to fill out and to present together with the passport at the border-check. The form was very similar to this one: http://www.uscis.gov/files/form/i-485.pdf

When somebody is signing up for a new account, you can ask to answer some question like:
1. Are you a bot?
2. Are you a spam-bot?
3. Do you sign up to spam the forum?
4. Are you going to copy others script?
....

PS: It's told that more than half of the nazi criminals were captured because they have answered yes to the 7th question on page 3 :)

Photodeus, very interesting. Anyone use stackoverflow? Has it worked out for them? (they just launched so they probably don't have issues yet)

Aquilax, I'm awaiting your i485 form, until then you are banished from the site ;p

I like that idea. I hate how new accounts are able to post so many times. I think they should be limited to X number of posts/comments per half hour (like 5) until they have enough points.

I just quickly read the stackoverflow, and it sounds like a really good concept. I especially like this idea...

At the high end of this reputation spectrum there is little difference between users with high reputation and moderators. That is very much intentional. We don't run Stack Overflow. The community does.

Anyhow, one site that I occasionally visit (HotUKDeals) has a reputation system that seems to work so you might want to take a look. Basically users can post deals that can be judged as 'hot' or 'cold' by other users, and users can get reputation points also (based on deals or comments). I guess there could be some analogy between deals and scripts if a similar system was used here. There are details on the temperature system here, and the reputation system here.

Hmm, I just posted this to the spam section having not seen this topic, but I'd recommend looking at Kuro5hin and Slashdot. Slashdot is overtly simplistic because they need to handle the massive load that their readership generates, whereas Kuro5hin does cool things like letting you suggest scores directly and then averaging the suggested scores (rather than giving users an occasional +1/-1 swing).

As to karma - Slashdot has a great karma system (for its purposes) formulated around the net modification to your posts' scores plus 3 times the number of published articles you've submitted. The user must also have been active for a month or so. With karma >25, posts start at a +1. Since moderation is chosen randomly from the user base, higher-karma users do not moderate more strongly (under an averaging system like Kuro5hin, there is no default score. high-karma users could have their votes double-counted ... I don't know anything about Kuro5hin's karma system). Recall that most of slashdot's readership is not logged in; anonymous posts start at 0 points (instead of 1 or 2), and only users get the chance to moderate.

Userscripts.org might have a different set of thresholds for karma that might dictate multiple levels. I'd think something like for every ten installs of your scripts, you get a point (up to 15) and your net post ratings also applies (not counting karma bonuses!) with <s>no upper limit</s> (update 10/8: there MUST be an upper limit, or a karma whore (see below) can safely post trash. The upper limit should be 1.5x to 2x the higest threshold, so 30-40, and don't publish the number!). Karma thresholds could then be 0, 5, and 20 for "green user/scriptwright," "junior user/scriptwright," and "veteran user/scriptright" (or whatever you'd like to call them). Timing thresholds would be something like one month for Junior and one year for Veteran. A script author commenting on his/her own script (or the topic-starting user) should be treated as veteran for that topic. Greens would not be able to moderate and would post at a default score of 0, Juniors would be able to moderate (+1/-1) and would post at a default score of 1, and Veterans would be able to moderate twice as strong (+2/-2) and would post at a default score of 2. This should not be implemented verbatim ... lots of consideration will be needed here. An averaging system would have to have higher thresholds. Also note, Slashdot allows you to opt out of your karma bonus on a single post, though I'm not sure how wise that is.

Rating scripts might prove useful too. I'd probably treat them as comments (1x), maybe 2x. Rated scripts should almost certainly be by averages. In that case, with a +1/-1 comment scoring system, the weighting takes care of itself (since a 1/5 script isn't so great, but we don't want to penalize you for it, and a 5/5 script would be +4 points, which is fine by me).

Thanks for the awesome comments khopesh!

One slight concern is the number of installs is currently very game-able. (every http request against the script increases the count). Perhaps I can implement a onclick handler that registers when logged in users install a script and it counts towards a different install count which has the point system you recommend.

Also ratings of scripts are coming back. I hope to get to it relatively soon.

Yeah, that's a problem with the proposal. I've been quite surprised that you don't already do that; since I have several systems, I account for 3-5 of the install count on each of my scripts. Don't count installs by the script's writer (or any IP the scriptwright has recently logged in on), and keep install counts unique per IP (or maybe a /29 CIDR IP block if you're worried about computer labs and aggressive DHCP renewals). Limiting the karma from the install count to unique user accounts is fair too, but it might make for seriously low numbers.

Due to gaming the system, Slashdot decided to remove the karma points from a user's settings (they replaced it with descriptors like good, poor, excellent, etc ... I don't know them because I was already maxed out), and they never published the actual karma thresholds, though it could be figured out given one's points.

I highly recommend not showing a user his/her score and following Slashdot's model here. Just tell users that they get rewarded for good posts and good scripts, and give a vague indicator somewhere (so maybe the green/junior/veteran levels have sub-levels that hint at one's progress towards veterans). Anybody who really digs will find this topic in the forums and can guess based on what was said here.

Further reading: Karma Whoring (another form of gaming the system that is much harder to regulate) and karma on Slashdot's FAQ. (I updated my earlier post to reflect that there must be an upper limit to karma or else there's too much padding for free trash talking and spam, and I created a new topic to talk about a bug that inflates install counts.)