Superior Searching
Posted by Jesse Andrews on Oct 20, 2007
Searching for userscripts used to be slow and only search the descriptions. Now searching uses SOLR and searches names, summary, descriptions and source.
Searching on this site always sucked, for you and me... Every day I get a report of the the performance of the site (rails performance analyzer). I always see:
Slowest Request Times:
ScriptsController#search took 69.622s
ScriptsController#search took 64.092s
ScriptsController#search took 55.131s
ScriptsController#search took 49.324s
ScriptsController#search took 48.399s
ScriptsController#search took 47.859s
ScriptsController#search took 45.436s
ScriptsController#search took 44.657s
ScriptsController#search took 43.686s
ScriptsController#search took 42.745s
70 seconds for a search!!! That is bad for everyone - who wants to wait for over a minute for results, and I have to make it scale. Luckily the average search time is less than a second (here are the stats for the last day):
Count Avg Std Dev Min Max
ScriptsController#search: 12366 0.728 2.256 0.128 69.622
I had been hoping Henrik would have time to add acts_as_solr to the site since he is doing lots of cool stuff with it on other rails projects, but alas, he is busy. So I rolled up my sleeves, and released solr based search.
The search order is currently only ranked on the prevalence of the search in the document (ignoring number of favoritings, installs, comments, ...) but it is a start. Once this is stabilized adding better sorting shouldn't be too hard.
Now if only the front page didn't suck so bad... ;)
You could comment on this post if you were logged in.
login to vote
Good work! Thanks, Jesse.
login to vote
Great news. Searching before did indeed suck! Off to try it out now...
login to vote
Cool, thanks. That would be great if it gave the option to sort by the other criteria you mentioned as well!
login to vote
The problem now is the way the results are sorted out. It was better before when they where ordered from most recent to oldest updated script. Now I see scripts from 2006 next to the ones updated only hours ago.
Hope you guys fix this.
login to vote
"The problem now is the way the results are sorted out. It was better before when they where ordered from most recent to oldest updated script."
I think so ! impossible to see if there's new or update script !
login to vote
I've been reading on sorting options - right now it is sorting by "score" - I'll add in sorting by create / updated date
login to vote
Search results are now ordered by the most recent updated source first
login to vote
Wonderful!! Thanks!
login to vote
hmmmm, spam
login to vote
thank you. very good news.
ersinlogin to vote
Good work guys ..but the all the script headings are not posted in english...it would be better if it is available in english too