An awful lot of my web site is driven by cgi scripts. Sometimes I introduce bugs when I change something, but I usually notice that in my testing.
Recently I noticed some unusual errors in my web logs, and although I knew that they were related to a particular flat file that has information about pages, I couldn't figure out which script was triggering the errors.
What I was seeing was lines like this:
/usr/home/pcunix/www/data/control/Unixart: 40: Syntax error: word unexpected (expecting ")")
In the first place, that file shouldn't have been marked as executable, but of more interest to me was why any script was even trying to execute it. That meant that I must have been using a Perl "qx" command to call a shell script, which is something I rarely do, but unfortunately it is in enough scripts that I couldn't tell for sure where the problem was. The problem could be in any page, because each page is littered with SSI includes. It's not easy to track because a general script may call something more specific depending on what page it is, etc.
So, back to the logs. People do make enough other typing errors that I had a bracket of time stamps within which these errors had to be. Unfortunately, the smallest bracket was still several minutes, and that represents a lot of accesses. I cut out the 1800 lines where the problem had to be, and used vi to eliminate image accesses, leaving only pages and direct calls to scripts. I eliminated all unneeded text, leaving only the page or script, and then ran it through "sort -u" to get rid of duplicates. The file now looked something like
... /Unixart/winauth.html /Unixart/winrip.html /answers.html /cgi-bin/printer.pl?/Bofcusm/2305.html /currentpatch.html /index.html /newtosco.html /robots.txt /search.html ...
I started a "tail -f" on the error log, and then ran this:
for i in `cat stufffromlogfile` do echo $i lynx -dump https://aplawrence.com/$i > /dev/null sleep 6 done
In a few minutes, I saw the errors pop into the error log, so I tried the last file name displayed by the script, and sure enough, that was it. Tracking to the faulty script was then easy. The script was doing something like
$cmd="/usr/bin/grep $this $thatfile";qx($cmd)
and under certain circumstances I needed to "chomp $this" and had not.
Got something to add? Send me email.
More Articles by Tony Lawrence © 2009-11-07 Tony Lawrence
The easy confidence with which I know another man's religion is folly teaches me to suspect that my own is also. (Mark Twain)