Small Performance Test Rails vs. CakePHP

At my company we're currently evaluating a Web-Framework. Today we had a look at CakePHP and got some simple tutorial examples running. Because CakePHP is very close to Rails, I wrote one example also in Rails to see differences.
Just to get an impression about the performance of the two frameworks, I run ApacheBench to get some benchmarks.

But first things first, here some figures about the test environement:

• iMac OS X 10.5.6, 2 GHz PowerPC G5, 1 GB RAM
• PHP 5.2.6
• CakePHP 1.2.0.7962
• Ruby 1.8.6
• Rails 2.2.2
• MySQL 5.0.67

I wanted a very simple scenario, but one that effects all MVC components. I chosen to take an overview page, that displays a list of objects stored in the database. With Rails this task was accomplished by just using the scaffold generator. With Cake I could have also used the scaffold variable, but then I would have compared a static scaffolded page against a dynamic one. So I wrote the index function:

function index(){
  $jobs = $this->Job->find('all');
  $this->set('jobs', $jobs);
}

I put only 3 rows in the database table, so that reading from it takes virtually no time and we can better see the time consumed by the framework itself.
As one Mongrel can only handle one request at the time, I fired 50 requests in a row.

ab -n 50 -c 1 http://127.0.0.1:3000/jobs

Last thing I made, was a very small and straight forward PHP script, that does the same job. This way I had a benchmark of a script using only plain vanilla PHP with no framework overhead.
Here finally the results (see full reports below):

CakePHP                       Time per request:       365.572 [ms] (mean)
Rails (RAILS_ENV=development) Time per request:       105.029 [ms] (mean)
Rails (RAILS_ENV=production)  Time per request:        30.318 [ms] (mean)
Pure PHP                      Time per request:         9.026 [ms] (mean)

Honestly I was quite surprised. There's so much talk about Ruby being slow, which is surely true when compared to Java. Also PHP is faster in microbenchmarks. But it seems to me that PHP is only really fast when used as much as possible straight forward, this is what PHP was designed for. When it comes to build frameworks and OOP there're better tools.
Also the other known PHP Frameworks CodeIgniter and Symfony are not that different, CakePHP lies somewhere between them.
On the other hand Ruby really was designed as a pure OO language and it seems that this turns to be an advantage when building frameworks, also from a performance perspective.

The full results:

Rails (RAILS_ENV=development)
=============================

ab -n 50 -c 1 http://127.0.0.1:3000/jobs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        Mongrel
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /jobs
Document Length:        3043 bytes

Concurrency Level:      1
Time taken for tests:   5.251 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      178251 bytes
HTML transferred:       152150 bytes
Requests per second:    9.52 [#/sec] (mean)
Time per request:       105.029 [ms] (mean)
Time per request:       105.029 [ms] (mean, across all concurrent requests)
Transfer rate:          33.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       6
Processing:    74  105  64.5     83     318
Waiting:       73  103  63.6     83     317
Total:         74  105  64.7     84     318

Percentage of the requests served within a certain time (ms)
  50%     84
  66%     86
  75%     88
  80%     91
  90%    237
  95%    304
  98%    318
  99%    318
 100%    318 (longest request)

RAILS (RAILS_ENV=production) 
============================

ab -n 50 -c 1 http://127.0.0.1:3000/jobs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        Mongrel
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /jobs
Document Length:        3043 bytes

Concurrency Level:      1
Time taken for tests:   1.516 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      178236 bytes
HTML transferred:       152150 bytes
Requests per second:    32.98 [#/sec] (mean)
Time per request:       30.318 [ms] (mean)
Time per request:       30.318 [ms] (mean, across all concurrent requests)
Transfer rate:          114.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       3
Processing:    17   30  32.8     21     187
Waiting:       16   29  32.8     20     186
Total:         17   30  32.8     21     187

Percentage of the requests served within a certain time (ms)
  50%     21
  66%     24
  75%     25
  80%     26
  90%     34
  95%    138
  98%    187
  99%    187
 100%    187 (longest request)

CAKEPHP
=======

ab -n 50 -c 1 http://jobscake/jobs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking jobscake (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        jobscake
Server Port:            80

Document Path:          /jobs
Document Length:        4176 bytes

Concurrency Level:      1
Time taken for tests:   18.279 seconds
Complete requests:      50
Failed requests:        49
   (Connect: 0, Receive: 0, Length: 49, Exceptions: 0)
Write errors:           0
Total transferred:      217781 bytes
HTML transferred:       200781 bytes
Requests per second:    2.74 [#/sec] (mean)
Time per request:       365.572 [ms] (mean)
Time per request:       365.572 [ms] (mean, across all concurrent requests)
Transfer rate:          11.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       3
Processing:   329  365  42.2    355     570
Waiting:      329  364  39.6    355     543
Total:        329  365  42.4    356     572

Percentage of the requests served within a certain time (ms)
  50%    356
  66%    364
  75%    372
  80%    374
  90%    390
  95%    409
  98%    572
  99%    572
 100%    572 (longest request)

PURE PHP
========

ab -n 50 -c 1 http://localhost/develop/projects/Test/pure_php_jobs/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /develop/projects/Test/pure_php_jobs/
Document Length:        1923 bytes

Concurrency Level:      1
Time taken for tests:   0.451 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      107200 bytes
HTML transferred:       96150 bytes
Requests per second:    110.80 [#/sec] (mean)
Time per request:       9.026 [ms] (mean)
Time per request:       9.026 [ms] (mean, across all concurrent requests)
Transfer rate:          231.98 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       1
Processing:     5    9   7.3      6      53
Waiting:        3    7   7.7      5      53
Total:          5    9   7.3      6      53

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      8
  75%     10
  80%     12
  90%     13
  95%     16
  98%     53
  99%     53
 100%     53 (longest request)