RE: [TV] bleb.org needs to move

["Ivor Hewitt" <ivor@xxxxxxxx>]

> > 
> > Roughly how much CPU and bandwidth does the site consume, 
> and roughly 
> > how many hits/day does it get?
> 
>  * CPU is minimal on a 1Ghz VIA CPU (using mod_perl)
>  * Bandwidth seems to now be ~20GB a month (http://tinyurl.com/3j892
>    suggests somthing evil's happening at 3am)
>  * Same URL suggests nearly 2m hits[1] in December
> 
A quick scan through those stats suggests a few immediate changes may be
worthwhile:
A good Gb is from spiders, suggesting a few tweaks to your robots.txt file
would be in order, after all is there any need for the spiders to cache all
of your listings content?
Also you're chewing up 200Mb just serving 404 pages for pages that are being
programatically scanned. Probably worth just not returning anything at all
for non-existant /tv/data/listings files.

As for 3am... Hmmm depends how your totals are calculated. If the total for
3am is the total snapshot number of cumulated hits at 3am. i.e. hits that
have occurred between 2am and 3am.... Then perhaps the fact that the default
mythtv mythfilldatabase cron is kicked off at 2am might be a clue perhaps?

Finally perhaps it's time to consider ways of doing distributed xmltv feeds.

Cheers,
Ivor.