This website limps along serving up a little less than 200,000 pages a month. If you divide that up, it's a bit over 6,000 a day, which comes down to 4 or 5 pages per minute. That's hardly a lot to ask of any webserver.
Of course sometimes we get peaks. There's not much happening at 3:00 AM so obviously there's more happening at noon. Every now and then some article gets a little burst of popularity despite my best efforts to discourage such things. Those cause peaks too.
But lately we've been getting some plain old peaks. I mean that they don't come because a particular page has attracted some attention; they come just because a lot of people came at once. For example, we had one just a few minutes ago and it hit just about one page view per second. As each page also has to pull in other stuff, the load on the server was three times that and it started throwing errors:
"unable to include /cgi-bin/blah-blah.pl", "Premature end of script headers:", "Permission denied".. not good.
Well, you can sympathise. If that load kept up all day and all night, I'd be serving over 2 million pages per month and that might be a little too much to ask of a shared web server.
So what to do? Well, I could use mod_perl. That's annoying, but it's probably what I have to do. Unfortunately, my code is far from ready for mod_perl so that will take some time and effort.
I could create static pages. That would be even more annoying.
I could ask more of you to use RSS. Go ahead and do that. I don't think it will help that much.
I could switch to a more expensive server. I really don't want to do that either.
Any other suggestions?
Got something to add? Send me email.
More Articles by Anthony Lawrence © 2012-07-16 Anthony Lawrence
The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming. (Donald Knuth)