Small Performance Test Rails vs. CakePHP

At my company we're currently evaluating a Web-Framework. Today we had a look at CakePHP and got some simple tutorial examples running. Because CakePHP is very close to Rails, I wrote one example also in Rails to see differences.
Just to get an impression about the performance of the two frameworks, I run ApacheBench to get some benchmarks.

But first things first, here some figures about the test environement:

• iMac OS X 10.5.6, 2 GHz PowerPC G5, 1 GB RAM
• PHP 5.2.6
• CakePHP 1.2.0.7962
• Ruby 1.8.6
• Rails 2.2.2
• MySQL 5.0.67

I wanted a very simple scenario, but one that effects all MVC components. I chosen to take an overview page, that displays a list of objects stored in the database. With Rails this task was accomplished by just using the scaffold generator. With Cake I could have also used the scaffold variable, but then I would have compared a static scaffolded page against a dynamic one. So I wrote the index function:

function index(){
  $jobs = $this->Job->find('all');
  $this->set('jobs', $jobs);
}

I put only 3 rows in the database table, so that reading from it takes virtually no time and we can better see the time consumed by the framework itself.
As one Mongrel can only handle one request at the time, I fired 50 requests in a row.

ab -n 50 -c 1 http://127.0.0.1:3000/jobs

Last thing I made, was a very small and straight forward PHP script, that does the same job. This way I had a benchmark of a script using only plain vanilla PHP with no framework overhead.
Here finally the results (see full reports below):

CakePHP                       Time per request:       365.572 [ms] (mean)
Rails (RAILS_ENV=development) Time per request:       105.029 [ms] (mean)
Rails (RAILS_ENV=production)  Time per request:        30.318 [ms] (mean)
Pure PHP                      Time per request:         9.026 [ms] (mean)

Honestly I was quite surprised. There's so much talk about Ruby being slow, which is surely true when compared to Java. Also PHP is faster in microbenchmarks. But it seems to me that PHP is only really fast when used as much as possible straight forward, this is what PHP was designed for. When it comes to build frameworks and OOP there're better tools.
Also the other known PHP Frameworks CodeIgniter and Symfony are not that different, CakePHP lies somewhere between them.
On the other hand Ruby really was designed as a pure OO language and it seems that this turns to be an advantage when building frameworks, also from a performance perspective.

The full results:

Rails (RAILS_ENV=development)
=============================

ab -n 50 -c 1 http://127.0.0.1:3000/jobs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        Mongrel
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /jobs
Document Length:        3043 bytes

Concurrency Level:      1
Time taken for tests:   5.251 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      178251 bytes
HTML transferred:       152150 bytes
Requests per second:    9.52 [#/sec] (mean)
Time per request:       105.029 [ms] (mean)
Time per request:       105.029 [ms] (mean, across all concurrent requests)
Transfer rate:          33.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       6
Processing:    74  105  64.5     83     318
Waiting:       73  103  63.6     83     317
Total:         74  105  64.7     84     318

Percentage of the requests served within a certain time (ms)
  50%     84
  66%     86
  75%     88
  80%     91
  90%    237
  95%    304
  98%    318
  99%    318
 100%    318 (longest request)

RAILS (RAILS_ENV=production) 
============================

ab -n 50 -c 1 http://127.0.0.1:3000/jobs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        Mongrel
Server Hostname:        127.0.0.1
Server Port:            3000

Document Path:          /jobs
Document Length:        3043 bytes

Concurrency Level:      1
Time taken for tests:   1.516 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      178236 bytes
HTML transferred:       152150 bytes
Requests per second:    32.98 [#/sec] (mean)
Time per request:       30.318 [ms] (mean)
Time per request:       30.318 [ms] (mean, across all concurrent requests)
Transfer rate:          114.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       3
Processing:    17   30  32.8     21     187
Waiting:       16   29  32.8     20     186
Total:         17   30  32.8     21     187

Percentage of the requests served within a certain time (ms)
  50%     21
  66%     24
  75%     25
  80%     26
  90%     34
  95%    138
  98%    187
  99%    187
 100%    187 (longest request)

CAKEPHP
=======

ab -n 50 -c 1 http://jobscake/jobs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking jobscake (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        jobscake
Server Port:            80

Document Path:          /jobs
Document Length:        4176 bytes

Concurrency Level:      1
Time taken for tests:   18.279 seconds
Complete requests:      50
Failed requests:        49
   (Connect: 0, Receive: 0, Length: 49, Exceptions: 0)
Write errors:           0
Total transferred:      217781 bytes
HTML transferred:       200781 bytes
Requests per second:    2.74 [#/sec] (mean)
Time per request:       365.572 [ms] (mean)
Time per request:       365.572 [ms] (mean, across all concurrent requests)
Transfer rate:          11.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       3
Processing:   329  365  42.2    355     570
Waiting:      329  364  39.6    355     543
Total:        329  365  42.4    356     572

Percentage of the requests served within a certain time (ms)
  50%    356
  66%    364
  75%    372
  80%    374
  90%    390
  95%    409
  98%    572
  99%    572
 100%    572 (longest request)

PURE PHP
========

ab -n 50 -c 1 http://localhost/develop/projects/Test/pure_php_jobs/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /develop/projects/Test/pure_php_jobs/
Document Length:        1923 bytes

Concurrency Level:      1
Time taken for tests:   0.451 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      107200 bytes
HTML transferred:       96150 bytes
Requests per second:    110.80 [#/sec] (mean)
Time per request:       9.026 [ms] (mean)
Time per request:       9.026 [ms] (mean, across all concurrent requests)
Transfer rate:          231.98 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       1
Processing:     5    9   7.3      6      53
Waiting:        3    7   7.7      5      53
Total:          5    9   7.3      6      53

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      8
  75%     10
  80%     12
  90%     13
  95%     16
  98%     53
  99%     53
 100%     53 (longest request)

Ruby meets Smalltalk

Avi Bryant , der Autor des Smalltalk Webframework Seaside, hat dieses Wochenende an der RailsConf 2008 MagLev vorgestellt, eine VirtualMachine für Ruby. Gemstone, die Firma hinter MagLev, hat schon jahrzentelange Erfahrung mit Smalltalk VMs und hat dieses Knowhow nun für Ruby umgemünzt. Anders ist auch kaum zu erklären wie sie in so kurzer Zeit, die Rede ist von 3 Monaten, eine eigene VM auf die Beine stellen konnten.

Erste Demos und Vergleiche mit MRI scheinen den alten Intepreter schlicht und einfach stehen zu lassen, MagLev soll je nach Benchmarktest zwischen 6 und 111 mal schneller sein.
Allzu oft haben wir schon gehört, dass eigentlich nichts dagegenspricht, dass Ruby so schnell wie Smalltalk sein könnte. MagLev hat nun offenbar den Beweis angetreten.

Noch ist MagLev nicht fähig Rails laufen zu lassen, aber Avi Bryant scheint keine Zweifel daran zu haben, dass auch dies in Kürze erreicht werden kann.

Die Kehrseite an der neuen VM ist wohl, dass sie nicht OpenSource sein wird. Der Ruby Sourcecode wird zwar öffentlich sein, der C Code für den Compiler wird aber geschlossen bleiben. Voraussichtlich wird es verschiedene Lizenzen je nach Grösse und Umfang geben. Ist zu hoffen, dass MagLev auch in einer Version für kleinere Budgets zu haben ist. Denn viel wird genau davon abhängen, ob sich MagLev in der Rails Community durchsetzten kann.

Update: Der erste Teil von Avi's Präsentation:
MagLev presentation at RailsConf 2008 - part 1 from Monty Williams on Vimeo.

Roxanne bekommt Flügel

Kurz nach dem Release von Rails 2.0 kommt nun auch der neuste Release von Roxanne. In dieser neuen Version stand vorallem die Performance im Vordergrund:

Erstens wurde ActsAsTree durch ActsAsNestedSet ersetzt, genauer gesagt mit BetterNestedSet. Damit können nun ganze Content-Hierarchien mit einem Query geladen werden. Zum Vergleich, eine Seite mit etwa 250 editierbaren Elementen brauchte auf einem G4 1.42 GHz MiniMac im development Modus etwas über 9 sec (gähn!), im productive Modus etwa 3 sec. Jetzt braucht die gleiche Seite im development 6.5 sec, noch nicht gerade berauschend, aber im productive Modus nur noch 0.7 sec.

Zweitens setzt Roxanne nun auch Caching ein. Solange man auf der Site nicht eingeloggt ist, werden die Seiten direkt aus dem Cache geliefert. Das dauert dann gerade noch 0.0015 sec ;-) Und nicht zuletzt ist nun Roxanne auch Rails 2.0 ready.