Advanced Error Hunting

So what happens if I introduce logical errors to the script while I am debugging? Worse yet, what if there are 1000 lines of code and I am not sure where the error is because I was coding poorly and jumping back and forth through sections without constantly checking myself to see what I did?

Well, this is actually pretty common and there are quite a few ways to go about finding the error depending on your taste.

Command Line Tactics
The first and most common way to check to see where a script is failing is to run it from the command line because the command line will give you much more information than the web browser when you are trying to debug.

Perl makes it very easy for you to check the syntax of your CGI script by offering you a special "debug" mode. In order to check the syntax of your CGI script, simply type the following from the command line:

perl -c scriptname.cgi

The debug mode actually checks the syntax of your CGI script without actually executing the code. A listing of all of the debugging commands can be listed by typing

perl -h scriptname.cgi

Of course, if executing the code has no affects other than outputting, you can also just try running the script itself without debugging using the following command:

perl scriptname.cgi

Here is another one you'll see alot:

As you can see, Perl sends back a good deal of useful information about your problem. Typically, it will do its best to analyze what the problem was as well as give you a line number so that you can look into the problem yourself.

Well, as you may have guessed, there are quite a few commonly made syntax errors that will plague your command line executions. In fact, in "Teach Yourself CGI Programming in Perl", Eric Herrmann lists the following common suspects:

Symbol	Name	Description
;	semicolon	Each command in your Perl program must end with a semicolon. Unfortunately, the error message you get may not give you any hints. You'll usually get something to the effect of, "syntax error at 1.pl line 6, near print". It is up to you to track down the error.
{}	braces	Braces are used to delimit sections of the program (such as if, while or for loops). the most common problem is leaving off a closing brace to correspond with an opening brace.
()	parentheses	Every now and then, you will forget a parentheses in an if statement, just beware
""	Quotation Marks	Perl allows quoted strings to include multiple lines. This means that of you leave off a closing quote the rest of your entire program might be considered part of the quoted string. Also, beware of having quotes inside of quotes such as print "She said "hello""; How can perl know which quote is meant to be printed and which quote is meant to end the string to be printed?
@	At Sign	The @ character is used to name list arrays in Perl. Thus, if you are going to print an @ character such as when you print an email address, you must make sure to "Escape" it using a backslash such as print "selena\@eff.org";

Log File Analysis
Assuming that your system administrator has given you access to this file, another useful debugging tool is the error log of the web server you are using. This text file lists all of the errors which have occurred while the web server has been processing requests from the web. Each time your CGI script produces an error, the web server adds a log entry.

If your sys admin does not allow access to the log file, you may ask her to email you a version of the log file with only errors related to your work. She can create such a version by using the GREP command and it should not be too difficult.

On the other hand, if you do have access to the log file, it can usually be found in the "logs" directory under the main web server root.

For example on NCSA serves, it can be found at

/usr/local/bin/httpd/logs

Dressing up as a Web Browser
In "Teach Yourself CGI Programming in Perl", Eric Herrmann outlines a method which you can use to test your CGI scripts using TELNET. I recommend reading the section if you have the chance. In the meantime, here is a quick explanation...

If you are able to use the TELNET program to contact your web server, you can view the output of your CGI script by pretending to be a web browser. This makes it easy to see "exactly" what is being sent to the web browser.

The first step is to contact the web server using the telnet command:

telnet www.yourdomain.com:80

Once you have established a connection with the HTTP server, you formulate a GET request using the following syntax:

GET /cgi-bin/testscript.cgi HTTP/1.0

This command tells the server to send you the output of the requested document, which in this case is a CGI script.

After your GET request, the web server will execute your CGI script and send back the results which will look something like the following:

eff.org:~$telnet www.mydomain.com:80

trying 190.2.3.120 ...
Connected to www.mydomain.com.
Escape character is '^}'.
GET /cgi-bin/test.cgi HTTP/1.0
Hello World

Connection closed by foreign host.

Using print "Content-type: text/html\n\ntest";exit;
However, I will note a third method that you can use to find out where a logical error is when it is not a "syntax error" but an HTTP error. An http error causes the favorite, "404 document contains no data" error which the command line and error logs won't necessarily help with. The script will run fine from the command line, but it won't run from the web.

Look at the hello world script with a couple of minor changes

 
#!/usr/local/bin/perl 
require ("./Library/cgi-lib.pl"); 
&ReadParse(*form_data); 

foreach $incoming_form_variable (keys(%form_data)) 
  { 
  print "$incoming_form_variable =
    $form_data{$incoming_form_variable}\n"; 
  } 

print "Content-type: text/html\n\n"; 
print "Hello World"; 

When you run this script, you will get a "404 document contains no data" error because you have sent text to the browser (the variable names and values) BEFORE you have sent the magic HTTP header line "Content-type: text/html\n\n". But how would you find out that this is a problem.

The solution is to use the "print "Content-type: text/html\n\ntest";exit;" line to walk through your routine one step at a time to discover at which point the problem begins. let's try it.

 
#!/usr/local/bin/perl 
print "Content-type: text/html\n\ntest";exit;
require ("./Library/cgi-lib.pl");
&ReadParse(*form_data); 

foreach $incoming_form_variable (keys(%form_data)) 
  { 
  print "$incoming_form_variable =
    $form_data{$incoming_form_variable}\n"; 
  } 

print "Content-type: text/html\n\n"; 
print "Hello  World"; 

That is going to work just fine. The web browser will read "test" and we will know that the error is not being caused by the first line of the script. (Notice that because we use the "exit" function, Perl will stop executing the script so we will not get any of the other info.)

Next, let's move the testing line down...

 
#!/usr/local/bin/perl 
require ("./Library/cgi-lib.pl"); 
&ReadParse(*form_data); 
print "Content-type: text/html\n\ntest";exit;

foreach $incoming_form_variable (keys(%form_data)) 
  { 
  print
  "$incoming_form_variable =
    $form_data{$incoming_form_variable}\n"; 
  } 

print "Content-type: text/html\n\n"; 
print "Hello World"; 

That is going to work just fine too! I'm getting bold there jumping two lines at a time, but when you actually use this method, you can feel free to jump entire routines if you are sure they are not the cause of the bug. Just don't jump too many at once. Okay, now let's dump the line into the foreach loop.

 
#!/usr/local/bin/perl 
require ("./Library/cgi-lib.pl"); 
&ReadParse(*form_data); 

foreach $incoming_form_variable (keys(%form_data)) 
  { 
  print "Content-type: text/html\n\ntest";exit; 
  print "$incoming_form_variable =
    $form_data{$incoming_form_variable}\n"; 
  } 
 
print "Content-type: text/html\n\n"; 
print "Hello World"; 

Okay, That works too (remember to pass some variables as URL encoded data as shown above).

Finally, we move the line to the end of the foreach loop and we see that we get the 404 document contains no data problem!

 
#!/usr/local/bin/perl 
require ("./Library/cgi-lib.pl"); 
&ReadParse(*form_data); 
 
foreach $incoming_form_variable (keys(%form_data)) 
  { 
  print "$incoming_form_variable =
    $form_data{$incoming_form_variable}\n"; 
  print "Content-type: text/html\n\ntest";exit; 
  } 
 
print "Content-type: text/html\n\n"; 
print "Hello World"; 

That is it, we just discovered where the bug was. We can bonk ourselves on the head and say "Of course, the HTTP header MUST be the first thing printed to the browser!

Figuring out Where you are
Sherlock Holmes and the Case of the Broken CGI Script
In Conclusion