Thursday, November 8, 2007

The Great Merb Speedup

Well, Merb has been coming along in leaps and bounds, but unfortunately it's also slowed down a bit in it's rendering. With all the changes this method has become heavy and quite slow.

The merb team did some great work on this yesterday. Hopefully there'll be some more in the near future too. Here's some numbers for your delectation. Please take these with a grain of salt. I'm no expert on benchmarking. See below for a full setup of how I ran the tests.

This table is on pastie for your viewing pleasure

Rails: ab -n 5000 -c 5 http://127.0.0.1:3000/tester
Merb:  ab -n 5000 -c 5 http://127.0.0.1:4000/tester

Rails Version 8117
rake rails:freeze:edge
rake rails:update

Merb Version 

Invocation:
Rails: mongrel_rails start -e production
Merb:  merb -e production > a_log_file

=======================  RESULTS =========================
0 Partials

Merb 0.4.0:         Requests per second:    456.86 [#/sec] (mean)
Merb Edge 913:      Requests per second:    723.98 [#/sec] (mean)
Rails Edge 8117:    Requests per second:    375.47 [#/sec] (mean)

10 Partials
Merb 0.4.0:         Requests per second:    150.57 [#/sec] (mean)
Merb Edge 913:      Requests per second:    424.83 [#/sec] (mean)
Rails 1.2.5:        Requests per second:    214.91 [#/sec] (mean)
Rails Edge 8117:    Requests per second:    213.56 [#/sec] (mean)

100 Partials:
Merb 0.4.0          Requests per second:    22.42 [#/sec] (mean)
Merb Edge 913:      Requests per second:    100.83 [#/sec] (mean)
Rails 1.2.5:        Requests per second:    45.15 [#/sec] (mean)
Rails Edge 8117:    Requests per second:    44.23 [#/sec] (mean)

* Merb edge without view context caching on partials
* Experimental ONLY

0 Partials:         Requests per second:    727.22 [#/sec] (mean)
10 partials:        Requests per second:    573.50 [#/sec] (mean)
100 partials:       Requests per second:    229.13 [#/sec] (mean)

The setup - Rails

rails partial_speed_test
cd partial_speed_test
ruby script/generate controller tester

# setup AR to point to an emtpy db

# setup your controller See Below
# setup routes

mongrel_rails start -e production
# run the ab tests

The Setup - Merb

merb -g partial_speed_test_merb
cd partial_speed_test_merb
ruby script/generate controller tester
 
#setup your controller See Below
# Setup routes

merb -e production

Controller setup

# app/controllers/tester.rb
 
def index
  render
end

# app/views/tester/index.html.erb (RAILS)
Inside Index
<% 10.times do %>
<%= render :partial => "blah" %>
<% end %>

# app/views/tester/index.html.erb (MERB)
Inside Index
<% 10.times do %>
<%= partial :blah %>
<% end %>

# app/views/tester/_blah.html.erb
BLAH<br/>

Merb Tests

Procedure

  1. Uninstall merb gem

  2. Install merb gem

  3. Start app

  4. Start tests


Released 0.4.0 gem
svn co with revision 913 and rake install
change the partial method to use cached_view_context instead of clean_view_context(engine) in render.rb

Rails Tests


  1. Generate an app using the 1.2.5 gem and start it up

  2. Run ab tests

  3. rake rails:freeze:edge (8117)

  4. rake rails:update

  5. Start app and run ab tests

The number of partials was changed by changing the number in index.html.erb

Phew.. That was a long post.

Just to reiterate, take this with a grain of salt. This was just a test for my curiosity and I encourage you to run these yourself if you are so inclined. Hopefully I've captured all of the setup.

Update: Again with these numbers take them with a grain of salt. These numbers are from the activity monitor on OS X
Rails 8117          MEM(MB)         CPU       
10 Partials         24.9            99.7
100 Partials        25.8            100.0

Merb 0.4.0          MEM(MB)         CPU
10 Partials         15.6            98.8
100 Partials        21.4            100.0

Merb 913
10 Partials         14.3            98.8
100 Partials        13.8            100.0
Posted by at