CGI::Simple vs CGI.pm - Is twice as fast good enough?

After posting RFC CGI.pm refactoring a number of Perl Monks raised a very valid point - "Why bother?". CGI.pm is stable and its use widespread. Why bother changing from this? One 'good' reason might be speed. As a number of monks have requested speed comparison benchmarks with CGI.pm here are a range of them. Executive summary CGI::Simple is about twice as fast as CGI.pm depending on what you are talking about.

To use a module you need to load it, make an object, then extract stuff from that object. Depending on whether you are using mod perl or not you may or may not need a new process. Here are some tests:

Module Loading

To do the load test we need to trick perl into reloading the modules over and over which we do by by undefing %INC and requiring them in:

my ($start, $end, $cgitime, $simpletime);
my $n = 100;

$start = time;
do{require CGI; undef %INC} for 1..$n;
$end = time;
$cgitime = $end - $start;
print "Loading CGI $n times takes $cgitime seconds\n";

$start = time;
do{require CGI::Simple; undef %INC} for 1..$n;
$end = time;
$simpletime = $end - $start;
print "Loading CGI::Simple $n times takes $simpletime seconds\n";

__DATA__
Loading CGI 100 times takes 39 seconds
Loading CGI::Simple 100 times takes 23 seconds

As you can see CGI::Simple loads 79% faster than CGI.pm.

Extracting data from a CGI object

Now lets have a look at how fast we can extract data from our CGI object:

use Benchmark;
use CGI qw/:cgi /;
use CGI::Simple;
$ENV{'QUERY_STRING'} = 'foo=bar&baz=boo';
timethese(10000, { 'CGI'    => '$q = new CGI; $q->param("baz")',
                   'Simple' => '$s = new CGI::Simple; $s->param("baz")'});

timethese(10000, { 'CGI'    => '$q = new CGI; $q->param("baz") for 1..10',
                   'Simple' => '$s = new CGI::Simple; $s->param("baz") for 1..10'});

__DATA__
Benchmark: timing 10000 iterations of CGI, Simple...
       CGI: 22 wallclock secs (21.70 usr +  0.00 sys = 21.70 CPU) @ 460.83/s (n=10000)
    Simple: 15 wallclock secs (15.05 usr +  0.00 sys = 15.05 CPU) @ 664.45/s (n=10000)

Benchmark: timing 10000 iterations of CGI, Simple...
       CGI: 32 wallclock secs (31.20 usr +  0.00 sys = 31.20 CPU) @ 320.51/s (n=10000)
    Simple: 18 wallclock secs (18.57 usr +  0.00 sys = 18.57 CPU) @ 538.50/s (n=10000)

As you can see CGI::Simple is 43% faster making a new object and getting one param or 68% faster making a new object and getting 10 params.

Module loading and data extraction

In practical terms the module load time is often a choke point (depending on the application). Here is a test that tests module loading, object creation and parameter parsing together (note we don't completely undef %INC this time as we need to keep Benchmark.pm in %INC):

use Benchmark;

$ENV{'QUERY_STRING'} = 'foo=bar&baz=boo';

$cgi_code = <<'CODE';
%INC = ('Benchmark.pm' => 'C:/Perl/lib/Benchmark.pm');
require CGI;
$q = new CGI;
$q->param("baz") for 1..10;
CODE

$simp_code = <<'CODE';
%INC = ('Benchmark.pm' => 'C:/Perl/lib/Benchmark.pm');
require CGI::Simple;
$q = new CGI::Simple;
$q->param("baz") for 1..10;
CODE

timethese(100, { 'CGI' => $cgi_code, 'Simple' => $simp_code });

__DATA__
Benchmark: timing 100 iterations of CGI, Simple...
       CGI: 42 wallclock secs (43.50 usr +  0.00 sys = 43.50 CPU) @  2.30/s (n=100)
    Simple: 18 wallclock secs (18.56 usr +  0.00 sys = 18.56 CPU) @  5.39/s (n=100)

Unless you are using mod perl this testing is still inadequate as the startup time for a new process is not measured. This forms a significant part of serving a CGI request. Nonetheless there is a raw 134% performance improvement for using CGI::Simple over CGI.

New process creation testing

So, finally here is some data on the whole shebang: new process, load the module, make a new object and get some param data out:

C:\>type cgi.pl

$ENV{'QUERY_STRING'} = 'foo=bar&baz=boo';
use CGI;
$q = new CGI;
$q->param("baz") for 1..10;

C:\>type cgi-simple.pl

$ENV{'QUERY_STRING'} = 'foo=bar&baz=boo';
use CGI::Simple;
$q = new CGI::Simple;
$q->param("baz") for 1..10;

C:\>type test.pl

my $start;
my $n = 100;

$start = time;
`perl c:\\cgi.pl` for 1..$n;
print "$n iterations using CGI takes ", time-$start, " seconds\n";

$start = time;
`perl c:\\cgi-simple.pl` for 1..$n;
print "$n iterations using CGI::Simple takes ", time-$start, " seconds\n";

C:\>perl test.pl
100 iterations using CGI takes 73 seconds
100 iterations using CGI::Simple takes 40 seconds

C:\>

So testing the whole shebang CGI::Simple is just over 80% faster performing the same task.

However you want to look at this data it equates to being able to handle a lot more requests on the same server. To change from CGI to CGI::Simple is a one line change as the interface is identical.... Is it worthwhile considering? For me yes, for you who knows?

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print