Script Reviews
Posted by Jesse Andrews on Jan 23, 2008
The last straw happened this evening. Reviews are coming this weekend! Help plan them here.
The basics: unreviewed scripts CAN be installed by logged in users only... On the rails side I'll catch the request and send a 403 FORBIDDEN if the user isn't logged in.
Why? Because people are uploading scripts *cough* scrap all *cough* whose only purpose on the site appears to be a hosting service for them. They then use the userscripts as a component of a bookmarklet.
So - what does this mean?
Every script needs reviewed before it is public. If you are logged in you can see/install everything...
So, why has it taken this long? Well, the question I have is: Who determines if a script is acceptable? I neither can nor want to review hundreds of scripts a week. Obviously when a script is updated the changes will need to be accepted before they are public.
Are the reviewers a selection of a bunch of trustworthy users?
Do we make it so anyone can, but certain users have more weight (eg, the more karma you have the more your vote counts)? And add a delay between upload and acceptance if only "weak" users have rated it... giving time (a day?) for other users to see it and vote to reject?
UPDATE
I wrote this in utter frustration. After you comments and emails we implemented hiding of scripts with "dirty words" - see Hotlinking of Unlisted Scripts.
You could comment on this post if you were logged in.
|
|
Nice move...
|
|
|
user karma is a good idea, though i wonder if you could include a weighted system so that certain keywords (for instance, scrap all) are automatically given -X and would resist simple attempts a rigging the system. or instead of keywords, maybe parse the uploaded script itself and tag it as a 'scrap all' by noting a common 'signature'. i would also introduce a posting delay for new users - ie: newly registered users (maybe a two day probation?) cannot upload more than 1 script per day, or every hour. this will reduce the spam quite a bit too. |
|
|
Wouldn't this be pointless? As long as someone creates an account here, they'll still be able to use the scrap all scripts as bookmarklets. I think blocking hotlinking would be the most effective. I'm always logged in so will I still be bombarded with all of those scrap all scripts since I can see/install everything. I also see a problem with those same script creators adding in something extra that would automatically review/approve any scrap all script. If someone updates a script due to a bug fix, will it take 3 days for the updated script to be approved. Will you keep two copies of the script on the site during the review process? The original still being public while the updated version is being reviewed. |
|
|
I agree with nungryscpro. The scrap all problem seems to be the classic hotlinking problem. How about blocking request to *.user.js files based on the referer? If those spammers use USO to host the scripts for bookmarklets in orkut, their requests won't have a USO referer. The server could deny the request, redirect them to the script's page or even serve a simple script with an alert explaining that this use of USO is not allowed There are ways to spoof the referer, but it would help with the average user |
|
|
Hotlinking wouldn't be an issue for scripts that haven't been accepted since: 12345.user.js isn't a file on the filesystem, it is a route to a rails call. In the rails call I can see if the user is logged in or not, if they aren't they are sent a 403 FORBIDDEN. Scripts that are accepted could be "hotlinked". The reason I don't want to disable that is that there are valid usecases of grabbing the .user.js without visiting us.o first, and you cannot trust referrer (some firewalls remove it, and there are techniques for removing it via nested iframes) |
|
|
But, if I understand this system, a bookmarklet spammer could get the file even if it hasn't been approved if he/she is logged in. I fail to see how this will address the "scrap all" scripts flood. And, as nungryscpro said, a legitimate user who is always logged in will still have to bear those scripts |
|
|
This is something we've been thinking about for some time. One idea for a Review Board system is this: 1) The number of reviewers starts relatively small. Probably around 10 users or so. 2) User X gains karma by a number of means. Highly ranked form posts, excellent scripts, insightful comments, etc. 3) At some point user X is flagged as "Potential for Review Board," in which case USER X is reviewed by someone already on the board. 4) He get's accepted or denied, with the potential to be reviewed again. 5) Problem users can be handled in traditional moderation style fashion. Regarding logged in users being able to see the scripts still ... we of course already have the ability for a script to be marked as totally malicious and spam and hidden on the site, and a Reviewer could mark scripts as such. This also segments us into the extremely nice position of being able to offer scripts with "Stamps of Approval" that are known to be not only well written, but extremely useful for users. |
|
|
and what then if you need to pass a CAPTCHA before install and in this case hotlinking could be fixed, you don't need the referer check and so on. |
|
|
Britt, that sounds like a great idea! |
|
|
Perhaps obvious, but why can't we just simply disable hot-linking for all scripts? Why do we need to directly link to the .js? Force people to link to the /scripts/show/ instead. I suppose update scripts may want to hot-link, but tell them to get their own hosting too. My main script, gets updates from a secondary server. The upside to a secondary server, is that script updates don't count towards my install count, only initial installs. Summary: We could always just check the HTTP_REFERER header and disallow blank entries such as bookmarklets likely have. |
|
|
I've never paid attention to the 'scrap all' scripts, but if they are just using this site as a hosting service, I'll second the comment by KARASZI István.
|
|
|
I think what Britt is suggesting is spot on. I think that Stamps of Approval would help tremendously as, an end user, I do worry about using a malicious or defunct script. I think you should pick a trustworthy bunch of reviewers that are suited to job / qualified and definitely add a delay, just my opinion. |
|
|
Reviews, ok nice to have. But the main problem is, who does the job? Who hands out karma? On which basis? 'Highly ranked', 'excellent', 'insightful' - who decides about these classifications?
|
|
|
I like reviews and I'd love to see them imlpemented... but using it as means to stop scrap all scripts doesn't seem like the right way to take care of the problem. Using captcha sounds like a great option if it's feasible Regarding reviews if it's just a select few that do the reviews then what if they don't use the site the script is meant for. I've already seen that a couple orkut scripts were blocked that weren't scrap all scripts. I guess that's where it might fall into the communities idea. On the otherhand, if reviews are left up to the public (even just members) then we'll see biased reviews just like there are inflated install counts. We'll need reviewers to review the quality of the reviews. |
|
|
I don't write popular scripts. I write little scripts that do little things. I release my scripts to the public for the unlikely case that there is someone else out there who wants to do the same things. A review system would make it harder for people to find and install my scripts because most of my scripts wouldn't be reviewed, there just isn't enough interest in them. I think scripts should be assigned a score based on review, the score starting at zero. Any script with a score of zero or above should be allowed to be publicly installed/viewed. Scripts with negative score would disable hotlinking. Score would not start to be calculated until two reviews. A review could positively or negatively effect the score. In addition scripts could be placed into 3 categories based on score: Positive, Negative and None; None would be for scripts that didn't have 2 reviews. |
|
|
My suggestion: Add a CAPTCHA against bots and bored users.
Don't use a review system; to hard to check.
|
|
|
The result of manually reviewing each script will be less spam scripts being displayed but it's unlikely to have much effect on the number of spam scripts being uploaded, thus the amount of work to keep the site spam free will remain high. As mentioned earlier ff you also block hotlinking and redirect people to a message explaining the situation it should help to educate the orkut users and hopefully reduce the amount of spam scripts that are being uploaded. |
|
|
Hotlinking - don't block empty referer, just block specific sites. |
|
|
Any community-oriented review system will only be abused in my opinion. The answer seems simple:
If you create a system where a user can have "more weight", those with the fake accounts will simply boost the point total of whichever fake accounts they wish to make into "heavyweights". Every site that institutes a "karma" system, (or a "you'll get a cookie if you wag your tail when I say so" system) has these problems and quite frankly, it's the type of thing that these brainless kids on orkut would probably want. |
|
|
jk- I understand your concerns and I have the same! That is why we've not implemented it yet - I want to make sure it isn't breakable. |
|
|
How to make a script Im new here so I dont know that |
|
|
its very nice move ,good idea |
|
|
Thank you very much! i like this blog, too. |
|
|
Just test of my gravatar... |
|
|
Bad... It seems to be waiting |
