Find and replace

Subscribe to Find and replace 7 posts, 3 voices

 
Raytaylor User

Hi everyone.
I have been looking at various other scripts and cant seem to work one out to get this going.

I have a proxy server at home with a rather large cache, and free broadband internet during off-peak hours.
I would like to use a spider program to visit some websites and download the flv videos during off peak hours so they are in the proxy cache, so that when a user visits the website and looks at (for exsample,) the top 10 page, the videos were cached the night before and can be loaded from there.

This plan works well except that when the flash player requests the flv file, it adds a ? onto the url so its http://something/videofile.flv? and the proxy server wont load the flv file from the cache.
I have looked at the websites and in the html code it specifies the auctual flv file url to the flash player, but the flash player makes the request with the ? on the end.

What I would like to know is can a plugin be made where in the source code I could specify the start of the flash players code, up to the part where the url starts, so from ... to flv.value=" and then replace the code with something else.

I would then be able to use 2 greasemonkey scripts to replace the code before and after, with MP player i think its called which i have tested and works?

There are 2 main video websites that my flatmates visit so I can manually find the code to replace, but i have no idea how to make the script for grease monkey. My programming abilities go as far basic on an atari 400 lol.
So if i can find someone who knows how to write the script, i can insert the from / to parts of the code and what to replace it with.

I think this is a well requested script but no one seems to have any responses to past requests.

 
Raytaylor User

ps. I have tried url rewriting at the proxy server but it wont work because the player that these video sites use has a loading icon that stays in the middle of the screen while it plays so I need to use the alternatve player that i cant remember the name of at the moment.

Also there are a few random numbers that need to be removed to the removal needs to be a 'start removal at' and 'finish removal at' then 'replace with'

 
Mikado Scriptwright

Do you realize you will have to install the script on every computer behind your proxy? But what if someone uses another browser? I think you'd better tune the proxy software somehow.

 
Aquilax Scriptwright

Normally web-spider are stand-alone programs which send http request to the servers and parse the text reply. Some inferior web-spiders use IE as component to execute the http requests, that result in a 10 times slower spider with also other very annoying side effects, like javascript errors on pages which block the spider. But as far as I know you can't use FF as web-spider, or I'm wrong?

 
Raytaylor User

That is 12 computers across 4 flats, and everyone here uses firefox anyway so its no big deal.

 
Raytaylor User

The web spider i was going to use is "Website Ripper Copier". The program is able to extract the flv links from websites and download them to a local hard drive. In doing so, the proxy also caches them. Next time a web browser user visits the video, the proxy will dish it out from its cache. The website copier's copy just gets deleted when its downloaded and is only used to simulate previous browsing for the proxy.

 
Raytaylor User

Sorry i think it may use ie to make its requests however it seems to work. I have tested it by using the spider program on the site's top 10 page and then video pages linked from there. I then opened the html files, manually adjusted the code in notepad to the alternative player that doesnt add the ? onto the end of the url and saved it. Opened up in firefox and it worked - loaded from the proxy cache which i saw by looking at the network graphs on the proxy server - the LAN spiked to 4mb while the wan stayed at 0 where as they usually would both rise to 50kb/s

I have everything working, except the last piece of the puzzle which is using greasemonkey to do the find and replace on the fly, rather than me doing it manually in my testing.