Userscripts.org Blog

Exposing reviews and ratings

Sun, 31 May 2009 23:49:20 +0000

When browsing userscripts, you now have the option to sort by their average rating. Also scripts must have a sufficient rating before they are shown on the homepage.

In the 5 months since reviews were added to scripts, you guys have added 2372 ratings. To highlight cool scripts, I've added sorting of the scripts by their rating.

Right now the rating is the average of all reviews (without any weighting). This leads to two issues: 1) a highly script with a single bad review (60 five star ratings and a single 1 star) will come after a script with a single five star review. 2) reviews that the community vote are unhelpful (10 people vote the review is unhelpful) are included in the rating.

Now that reviews are exposed I expect the number of reviews to increase, and to iterate on the ranking algorithm to fix the mentioned issues.

Introducing Jetpack Support

Wed, 27 May 2009 01:51:53 +0000

Mozilla Labs recently introduced Jetpack, a new way of improving your browser using javascript/html. I'm introducing basic support for Jetpack on userscripts.org

Jetpack lets you extend your browser using JavaScript, much like Greasemonkey. We are happy to announce support for the platform on userscripts.org!

I've written a Private Message Notifier for Jetpack and shared it. I look forward to seeing what you guys create!

As people add Jetpacks, you will be able to find cool jetpacks

Becoming social

Tue, 12 May 2009 07:00:54 +0000

I've added vanity urls. So instead of being http://userscripts.org/users/2 you can be http://userscripts.org/users/anotherjesse (well, you cannot be anotherjesse since I already am). The beginning of a private messaging system is in place. And more email notifications are enabled.

Vanity URLs

The following people are paying attention and registered their vanity url already: 4got10, A.Porotnikov, Amarandus, Aquilax, Avindra Goolcharan, bd808, Beerallica, bennyboy2791, blastrix, bodi - jamal, bomexxx, Britt Selvitelle, C-a-n, cablop, ceres, Chad Boyce, Charles L., Charles R Greathouse IV, Cui Mingda, cxx, DaSilva, Decklin Foster, Defiant91, Deniz Burucu, Desen, Desmond Ng, drry, dupola, Eric S. Carr, fisio, GigeL, Goodguy, Gregory Eremin, HackVilla, HannibalSmith, hans212, HelenG, Henrik N, hkushi1216, Hypennne, iambran, Jesse Andrews, Joe Biellik, JoeSimmons, KaSeR15, Kevin Kuphal, KullDox, L3v3L, leandrobaldissera, loige, lunigma, Mamnun Hassan Bhuiyan, Manoel Neto, Marti, martin_english, Michael Howe, michele smorgon, Mike Gipson, Mindeye, Minh Truong, MrZeroo00, Nanobyte, navyjeff, North Union, Olley, omercnet, owen.pengtao, paopao, PennerProfi.de.vu, Penny Dreadful, Pete Prodoehl, Praveen Garlapati, Priorityman, ptown67, Pucek, pwlin, Raloc, Refael Ackermann, scizor93, Scott Jaeger, Scripts, Sean Canton, Seiromem, semarmendem, Semon, Shn, SirVek, sizzlemctwizzle, softforum, steeev, Studivz-Designs, svztalker, Sy r b, Sylvain Comte, Sythe, takunama, Ted Awesome, Tenkoji, ThatOneGuy, thedewil, thomd, tuxraider, uchari, X Enterprises, yak, yfdyh000, yishao, z33m.

The rest of you need to head to http://userscripts.org/home/settings and setup yours.

Private Messaging

The first release of messaging is enabled. If you are logged in you can see a link to your messages in the header (in the drop-down with your name). There is a link to send a message on a user's profile page. Send me a message and let me know what you think.

Once rate limiting and blocking is added (along with other things you guys find), I'll make it more visible on the site.

Email Notifications

Private messages are more useful when you know about them. So along with the ability to get an email when one of your favorite scripts are updated or when someone reviews one of your scripts, you can enable getting an email when someone sends you a private message. To enable these emails visit http://userscripts.org/home/settings/notifications as the default is to not send email. And yes, getting emails when reviews are written was enabled this evening as well.

Groups, Scripts and Voting

Sun, 03 May 2009 23:58:02 +0000

I've add the ability for group members to vote up/down on their scripts. This allows a group to rank their favorite scripts.

Groups are still a logged-in only feature (it appears on the header when you login).

Voting a script up/down only affects its ranking within that group. So if you vote down a script because it doesn't belong in the facebook group, it will not affect its ranking in the twitter group.

Voting on if a script belongs in a group is another step towards helping people find and share the best scripts.

The return of Web 2.0

Thu, 30 Apr 2009 17:59:15 +0000

Tags have been reworked, fixing many usability issues and adding some web 2.0 tag clouds to explore the site.

Tags are now saved in a way so that capitalization and special characters (periods, quotes, %, ...) are ignored. Now the tag Google Reader is the same as googlereader and GOOGLE/reader when browsing. The tags are stored as the script author enters then as well and shown when you are viewing a script.

The site homepage has a tag cloud of the most popular tags and other tags that have a good number of scripts.

The tags page has an even larger tag cloud. There is a link to browse all tags, which is a paginated list of tags sorted alphabetically or by number of scripts.

Previously the tag list was a 200K file of weirdly sorted tags (many of which were invalid).

Improvements to /users

Wed, 22 Apr 2009 08:54:14 +0000

The user index page has always added little value. I've started work on replacing it with an page that lets you see what is happening on the site. Click the users tab at the top to see it.

I've pulled in the hot threads occurring in the forums and script pages.

Also you can see the most active posters. You can see I'm at the top for now. When we migrated from comments to discussions (threads), I moved all the old comments into a single thread and added a message explaining what happened. This caused my insanely large number. I'll work on fixing that now that it is visible.

Reviews, issues, ... What would you like to see on the this page?

Userscripts.org in 2008

Mon, 05 Jan 2009 06:11:10 +0000

A 2008 summary of site statistics, review of site improvements, and a look at groups and Greasefire. Want a t-shirts?

52,956,864 Pageviews in 2008

These page views are by real visitors (running JavaScript) and do not include downloads (scripts, images, or other static content), userscript updaters, ajax requests, spiders, and other bot traffic.

Over 1,400,000 dynamic requests per day

An average of 900 requests per minute (cpm is calls per minute) hit the rails code running the site (that is over half a billion dynamic requests per year). Pingdom lets me know when the server explodes (which happens at the worst possible times). The site has had less than 3 hours downtime over the last 3 months.

3.5TB downloaded, 0.5TB uploaded

The downstream bandwidth has averaged 971Kb per second for the year (1.4Mb per second since September!). If the site was hosted on Amazon's EC2, bandwidth would cost $445 for the year. I expect the bandwidth to double in 2009.

Constant battles with CPU

Red is BAD! To combat constant growth, there have been many rewrites and infratstucture improvements. Systems changes include sphinx for search, Phusion Passenger for app servers, Quantcast for public stats, and NewRelic for performance monitoring.

Site Improvements

I cannot stress enough that the help given by community members in the site forum has been invaluable. After a long day working on my startup, their contributions give me the energy to spend more time in front of a computer trying to keep up with them. I'm grateful for all levels of help, from reporting a simple bug, helping newbies in forums, to writing html, css, and userscripts to improve the site.

The forum system was rewritten several times: moving script comments to forums, dealing with performance issues, read/unread status rewrite, stylistic improvements, and more. There were 44,673 posts in 2008 (and that doesn't include all the deleted spam).

The site had several major stylistic changes. The major being a new site header and script page. The year also brought many stylistic improvements from members (suggested in the forum).

Reviews were added in late October as a way to help users find the best scripts. So far 702 have been written (with 331 written in December).

Instead of overwriting the data with each upload, storing older versions of scripts was implemented. This helps answer the question "what changed?" and allows the community to watch for funny business by malicious script authors.

There are 25,386 scripts on the site, of which 15,286 were uploaded/updated in 2008.

The Future (where is my jet pack?)

In 2009 I'm concentrating on features that help users sort through all those scripts.

Steve Krulewitz wrote an open source extension called Greasefire help users know when there are scripts on userscripts.org written for the page they are viewing. Join him in the greasefire forum to discuss this new feature.

I've been working on groups with the goal of making it easier to find, share and collaborate on scripts (the first version released yesterday). A group can be centered around a site (NeoPets, Travian, Orkut, ...), concepts (using xpath), or a place for people to collaborate. The initial version of groups include members, forums and script collections. As the feature matures I hope to feature groups on the site homepage.

I plan on moving the entire site to Amazon's scalable infrastucture as I'm hitting the edge of what a single box can handle.

Last, I'm debating having a "userscripts t-shirts" made. Would you be interested in buying one? (probably $20 plus shipping for a high quality shirts) Let me know if you have opinions on the design or would buy one.

I hope that 2009 is another great year!

More Downtime

Thu, 11 Dec 2008 18:06:24 +0000

The morning of December 11th, the site went down for a few hours. Back up - read more for info on what happened.

I woke up this morning early to head down to Add-on-con. Unfortunately I got an SMS from pingdom that userscripts.org was down at 7:16AM PDT.

I was on the phone with ServerBeach at 7:20 after the automated reboot didn't work. They said they weren't able to talk to the server either and dispatched someone to the data center. A few minutes ago I got the note that it was up. I sshed in and started up the app.

Worst case scenerio: as I blogged on S3 Saves the Day, I had a backup of the database less than 12 hours old. So if the server was completely dead we would have lost a few hours of data but the site would have survived.

Sorry the frustration - it has been a frazzled morning for me as well :-/

Getting stylish!

Mon, 24 Nov 2008 03:45:03 +0000

This week has brought a new site header, script header and lots of other improvements. Thanks to all of those who helped make it happen!

Most importantly I want to thank avg whose bio on his user page was a script tag. The HTML sanitizing code wasn't being used for the bio field until a few minutes ago. If you find security holes in the future please email me directly before posting them on the site.

The site forum has had lots of activity recently with style, grammar and xhtml improvements. Script authors implementing improvements as userscripts is particularly helpful (recent examples by khopesh, pwlin and jerone) since it allows easy testing and comments from other users.

Site performance has been good (in my opinion), but the server is on the edge of not being able to handle the traffic. I'm working on improvements behind the scene to the caching mechanisms as well as moving to multi-server setup before the end of the year.

Future improvements will focus on the forum (which has layout issues), the rating system (quick thumbs up/down rating of a script to complement detailed 5 star reviews), and a group/categorization. I'll write more about those in the future.

Comments are dead, discussions are live!

Mon, 17 Nov 2008 00:50:15 +0000

Commenting on scripts has been popular, there has been 53,095 comments on 8,218 scripts since June 20th, 2005. Their success has come at a cost of usefulness. To bring order back to comments, comments are now threaded (eg, topics and posts).

All the existing comments have been archived into a locked thread (this is so people respond in new threads).

Now there can be different conversations about translations, ideas, bugs, and so on.

Lots of improvements are needed: the script author should have their posts highlighted, we will need a method of community based moderation, search, feeds, ... But like the introduction of reviews, I want to do it in steps.

As always you can join us in the forum to discuss the change and now you can follow @userscripts to talk with me about userscripts.org issues.

Fans, Reviews and Preview of "Guides"

Mon, 27 Oct 2008 05:03:27 +0000

Searching and browsing script by number of fans is now possible, reviewing/rating scripts is back (after a really really long hiatus), and a preview of "Guides".

My wrists have been really hurting but I took a bit of time and released a few fixes/features that have been nearly done for a while.

The previous work moving to sphinx included improvements to searching by number of fans a script has. I've deployed the frontend code so you can actually use it. The fans code might have issues (as I think it messes up when a user is deleted), so if you see anything, let me know!

In the spirit of helping users find the best scripts, I've decided to deploy ratings/reviews of scripts as it current is. I want to add ratings of reviews (5 of 7 people found this review helpful), so that the best reviews are highlighted. Rather than wait another week (or months), I want to get it out and see if there are any issues with the basics. The content of a review has a rating (1 to 5 stars), a title, and the actual review. The review can contain the same HTML as a comment - except no inline images.

One last thing that I've been working on that is still experimental is the concept of a "Guide". A guide is an article written by users that helps others improve their web. A guide can explain how to make the best use of a single script, or how to use several scripts work together to perfect the browsing experience. Guides currently are only visible to logged in users (this will change once the feature is finished and more guides are written).

I wrote a guide explaining what guides are. You can write your own or read guides written by others. The "Guides" feature need to be include features like commenting and ratings before being opened to the public (join us in the forum to discuss the feature)

Sorry that all of these features need polish, but I hope they are useful in state they are in.

New Search Engine

Wed, 22 Oct 2008 09:42:42 +0000

I've updated the search engine, speed and flexibility!

While search quality still needs improved (searching for flickr shows all those scripts that have images that come from flickr), the new system is much faster and works via updates in the background.

Updating a script used to take up 10 seconds (or more). Now it should be much faster.

The change was moving from SOLR (a Apache project that is a HTTP frontend to Lucene) to Sphinx. With SOLR after every update it would try to update the SOLR index before letting your request finish. Sphinx on the other hand runs in the background and re-indexes all the scripts every 5 minutes. This means changes take a few minutes to show up in search results, but the site runs much faster.

Times they are a changing

Wed, 08 Oct 2008 08:01:53 +0000

A new frontpage, integrate a how-to, integrated "Share this", made some style improvements, and the dashboard (/home) have been released!

The homepage has been revised, but is still not in a finished state. As more features get unveiled it will continue to evolve. So those of you who write scripts against userscripts.org, please don't spend too much time on it, instead talk about what it should have in the forums.

To those help in the forums, repeatedly answering the same basic question, I announce Userscripts and Firefox, a small how-to that is prominently linked from the homepage and right next to the INSTALL button on script pages. Let me know if you have improvements (or to contribute text for Opera/Safari/IE).

Share this has been added to individual script pages. I'm not sure if addthis or something based on socialhistory.js would be better. Opinions?

I've revised the comments CSS so that they take up the full width (along with being a little tighter). Also the site layout has had a few slight tweaks. It still looks great in my favorite alternate theme Dark Blue.

I've pushed more changes to your dashboard. From there script authors can see comments on your scripts, upload and update scripts. Monitored topics are now there as well - and monitored topics are private. Before anyone could look at your monitored topics list.

As always I'm sure there are lots of issues and I hope that you can help make it better! I've not had a chance to improve the points/karma system yet, but there has been some good discussion about it. I consider it a success so far as it has helped me prune comment spam when I have a few minutes here and there.

Voting on comments

Mon, 29 Sep 2008 03:46:01 +0000

To help highlight good comments and remove the bad ones, I've pushed a new version of the comment system.

Sorry to those of you whose userscripts break!

When you post a comment it automatically get "1 point". Any logged in user can vote thumbs up/down for that comment. When it falls below 0 points it the content of the comment will be hidden by default. (there is a show more link).

Also I've switched to using identicons as the default avatar if you don't have a gravatar.

Thanks to avg for styling suggestions of comments and JoeSimmons for helping with the front end for the new comment voting system.

Userscripts.org has no class

Wed, 27 Aug 2008 03:13:59 +0000

Users have been abusing the class attribute in comments/descriptions/other fields. It has gotten rather bad, so I've had to disable it :(

Sorry to those of you who were using the class attribute in useful ways, but the abuse has warranted this sudden change.

I think the class attribute should probably be legal in script descriptions, but limited to a preset?

Also commenters are using images in very aggressive ways, I think I'm going to kill that soon by making it so images in comments are not displayed until a user asks for them to be.

Performance Increases

Fri, 01 Aug 2008 08:47:40 +0000

Database, CPU and RAM usage cut in half. Slow parts of the site targeted. Thanks to New Relic.

I posted on my blog about improvements made on this site thanks to a Rails performance monitoring tool New Relic.

Long story short, viewing a script page should be much quicker now!

Looking at a chart of the slowest actions on the site we see that: updating a script, viewing the index of tags, and uploading a new version of the source of a script are the worst offenders.

Most of the slowness of updating the script is due to re-indexing into Solr. Currently that happens during the request cycle - leading to the user waiting while the search index updates. I'm planning to moving this to more of a queue based system.

I think the users and scripts index can be improved with an index as well, but will have to investigate this weekend.

Is there anything slow on the site that I (and New Relic) has missed?

Script Versioning

Tue, 29 Jul 2008 18:00:02 +0000

Versions are now stored for scripts. I'm working on "diff" functionality as well as adding "release notes" so users can see what has changed.

Let me know (email me) if you have any issues upgrading your scripts or installing others scripts.

Progress

Sun, 25 May 2008 07:00:35 +0000

Implemented several requests and am planning several other features/fixes thanks to your input.

I think the new feedback tab has worked out pretty good. I dislike that it requires a new user account with uservoice (the service that I'm using), but it was much quicker than implementing the service myself, allowing me to focus on fixing the site.

So far the fixed list includes:

unmonitoring of topics in the forum works
don't allow flash embeds in comments/forums
show the time a userscript was uploaded as well as the last update time
fixed logged exceptions rails plugin so admins can see when issues occur
comments are editable
users can delete their own comments
comment creation/editing uses forum bottom box style
a link to add a comment at the top of the userscript's comment list
updated links to beast (forum software) and famfamfam (icons)
remove users who haven't verified their email (unactivated) after 3 days
expose comment feed on userscripts page
expose feeds of updates to your favorite scripts and comments on all your scripts (see /users/me after you login)

I plan to keep fixing things as fast as possible, so keep telling me what needs fixed.

Feedback Redux

Mon, 12 May 2008 06:34:37 +0000

The site has been stable for a while, now lets start improving it.

Hi guys. My name is Jesse, and I apologize. I haven't been as responsive as I should with dealing with issues on the site. To that end I've integrated a service called uservoice that will help me build what you say is missing from userscripts.org!

Our feedback page at uservoice allows you to add issues and vote on which are the most important.

Hotlinking of Unlisted Scripts Disabled

Sun, 27 Jan 2008 13:10:34 +0000

Disabling hotlinking for unlisted scripts means they cannot be used in bookmarklets.

Scripts that are marked as unlisted are no longer hotlinkable.

To install them you MUST visit the script's userscipts.org page, where a unique link to the script is generated for each visitor. If you try to get the source of an unlisted script using a URL like http://userscripts.org/scripts/source/20989.user.js, you are given a 403 Forbidden page.

I've also posted a page about unlisted scripts. As we move forward this page will be updated (as will other pages about userscripts.org as we allow other users to edit them.)

Britt has implemented a badwords filter that will automatically unlist scripts that contain certain words (orkut or scrap for example). It will be activated Sunday. If you have a valid script that is unlisted due to the filter, contact us and we will relist it. This is an interim solution until we have the karma system in place.

UPDATE

The "dirty word" filter is in place. and it has been a day without having to review scrap all scripts! Instead I'm dealing with lots of emails from those users and having to explain disabling hotlinking doesn't break a Greasemonkey script - it breaks using it as a bookmarklet. So you script still works...