Friday, November 30, 2007

Raising Routeable Errors in Merb Router

On the #merb channel today there was an interesting puzzle to solve with routes. Merb routes have some crazy power, and today took my first crack at trying to give it a bit of a flex.

There were two distinct pieces of the puzzle that needed to be addressed. First, requests sent to actions on the application and exceptions controllers. The application controller will feed back with an error that says

"The 'Application' controller has no public actions"

The exception controller can be routed to like any other action. Failing missing actions are handled like any other controller, and actions rendering a known template will render but without the error message.

This behavior was not good for me and I didn't want it to occur. You might want the flexibility of being able to directly call the exceptions controller. But for this I don't.

Second, the error should be short and not give out too much information when either you request application or exceptions controller actions, or non-existent routes.

When Merb catches an exception it gives you a small amount of information about what the problem was. For this the requirement was that just a non-descript message be displayed.

This route setup prevents both issues from occuring and just renders an error screen with a simple message.

class Merb::Router
DEFAULT_NOT_FOUND_HASH = { :exception => Merb::ControllerExceptions::NotFound.new("URL Not Found"),
:controller => "exceptions",
:action => "not_found"}
end

puts "Compiling routes.."
Merb::Router.prepare do |r|

r.match(%r{^\/(application|exceptions)}).defer_to do |request, params|
params.merge!(Merb::Router::DEFAULT_NOT_FOUND_HASH)
end

# All the other routes

r.match(%r{.+}).defer_to do |request, params|
params.merge!(Merb::Router::DEFAULT_NOT_FOUND_HASH)
end
end
The first part of this route will catch any routes requested for application or exceptions controllers and render the error NotFound with the message "URL Not Found". The second part just catches any routes you haven't defined and renders that same error.

This setup will not work with default_routes. If you use default routes in between these to matchers, the bottom matcher will most likely not get called, and if you put the default routes after the last matcher in the example, default_routes will never be called. Personally, I don't like the idea of over riding the default behavior for not found errors, so I wouldn't have the second matcher, but I don't mind restricting routing from application and exceptions. At least that way you can use default_routes if you wanted to.

The defer_to method on a route will wait until the route comes in to assess it. If it matches the regexp then control is yielded to the block with the request and params objects.

Merbs error handling in controllers means that you just raise an error and the error will be caught by the exceptions controller and rendered. If you directly raise an error in the defer_to block though, your app will crash. You need to set the :controller, :action, and :exception options manually in the params hash to get error handling.

It's interesting to note that if the result of your defer_to block is false, it will not be considered as a match and will move onto the next. This does not allow for multiple defer_to matches to influence the result of another defer_to. For example, without default_routes set:
    r.match(%r{.+}).defer_to do |request, params|
params.merge!(:one => "one")
puts params.inspect
false
end

r.match(%r{.+}).defer_to do |request, params|
params.merge!(:two => "two")
puts params.inspect
false
end
will produce
{"action"=>"index", "one"=>"one"}
{"two"=>"two", "action"=>"index"}

# Instead of

{"action"=>"index", "one"=>"one", "two"=>"two"}
I'm not sold yet on what any of this means in terms of usefulness. I think I'll include the route to prevent requests going to application or exceptions directly, but I'll leave the normal behavior in for the rest of them.

Tuesday, November 27, 2007

New Merb Benchmarks

A while ago I started to do some really simple benchmarks on Merb and Rails. Yehuda Katz, aka wycats on IRC went and wrote two simple benchmark apps that live in Merbs SVN. To get these:
svn co http://svn.devjavu.com/merb/apps/benchmark 

svn co http://svn.devjavu.com/merb/apps/rails_benchmark
Both of these will use evented mongrel and swiftiply if they're installed.

To use these benchmark apps change into the root directory.
ruby script/benchmark
That command will run all the available benchmarks.

There are a number of options. Use the -h flag to help you find your way around. The options are there to help you setup the benchmark and also to select which benchmarks to run.
ruby script/benchmark -l
will list all the benchmarks available.
ruby script/benchmark -s erb,haml
will run the template and partials for erb templates and haml templates only.

Be careful though, these can take a while. Especially on Rails.

Organic Inefficiency

Merb really floats my boat at the moment. But I do think of other stuff... Really.

During my day job as a Mechanical Engineer it's part of my role to design and implement manufacturing cells in our factory. This can be pretty cool. Trying to make the cell as efficient and lean as possible can be quite a task sometimes. I've found that the cells not only need to be designed around the latest available (or is that affordable?) technology, people must be considered in the work flow if it's going to be successful.

When you design the cell with people being people in mind, all goes reasonably well. You can even make the cell efficient in a stand alone kind of way. But is that the best that can be done? I don't think so.

When the factory was first put on the floor, hopefully whoever did it did their homework and made an efficient layout so that all the individual work cells could function together reasonably well. As time goes on, you gain new products and you loose older ones. Some cells close and some new ones go in, and some of the bigger machinery which is very expensive to move now has to be planned around when you put in new equipment. What you end up with is an organically changing factory layout.

Picture this organically changing factory going forward for a few projects. Potentially you could end up with an unsatisfactory performance of the factory as a whole but this is hard to pick since all the individual cells seem to be efficient.

A work cell itself can have a very efficient theoretical process flow. In reality, if you can't translate that onto the floor the way you want, inefficiencies will result. The fact that when you lay the cell down, you "must put it in this spot" because that's where there's some free space can impact dramatically. Material in and out, and even flow in nice lines may not be optimum because you need to fit into the current factory layout. This can potentially lead to increased WIP (work in progress), increased handling equipment and all sorts of other nasties that may or may not be easily quantifiable.

It's my opinion that this kind of thing should be avoided. There is at least one software product I know of that's designed to assist with this issue. Tools like this can be very useful both to correct failing factories and also to move forward. Using a tool like FactoryFlow, is time consuming up front, for larger factories though it's much more expensive if you have to move equipment two or three times than if you invest up front with the design and do it right just once.

This is a huge issue that I'm sure could be talked about and theorised about. But this is a blog post so I won't do that to you :P

I think these same kind of organic inefficiencies can affect software development. I've seen plenty of old bloat-ware type products that attest to this. Automated BDD could go a long way towards alleviating this issue since the behavior is spec'ed and ideally you can change the implementation under the covers removing this organic inefficiency. Does it though? Is this kind of inefficiency common in software projects for the web?

I'd love to hear feed back on this issue.

Sunday, November 18, 2007

Native Sass Support for Merb

Nathan Weizenbaum aka nex3, a contributer to the awesome haml template system has just put support for using Sass in merb into the stable branch of Haml.

Haml has been supported in merb for a long time, longer than I've been involved with merb for sure. Sass was lacking though... Until Now :)

Before you can use Sass in Merb, you'll probably need to get the latest Haml.
sudo gem install haml
To use Sass in your Merb App it works the same way as in Rails. Put your Sass templates into public/stylesheets/sass
public/stylesheets/sass/my_style_sheet.sass
and include the css in the head of your layout application.html.*
css_include_tag :my_style_sheet
The only thing left to do now is to declare the dependency. In
config/dependencies.rb
include the line
dependency "haml"
Your app will now happily use your Sass templates, updated when you change them in development, but compiled only once during production. Sweet :)

Thursday, November 8, 2007

The Great Merb Speedup

Well, Merb has been coming along in leaps and bounds, but unfortunately it's also slowed down a bit in it's rendering. With all the changes this method has become heavy and quite slow.

The merb team did some great work on this yesterday. Hopefully there'll be some more in the near future too. Here's some numbers for your delectation. Please take these with a grain of salt. I'm no expert on benchmarking. See below for a full setup of how I ran the tests.

This table is on pastie for your viewing pleasure

Rails: ab -n 5000 -c 5 http://127.0.0.1:3000/tester
Merb: ab -n 5000 -c 5 http://127.0.0.1:4000/tester

Rails Version 8117
rake rails:freeze:edge
rake rails:update

Merb Version

Invocation:
Rails: mongrel_rails start -e production
Merb: merb -e production > a_log_file

======================= RESULTS =========================
0 Partials

Merb 0.4.0: Requests per second: 456.86 [#/sec] (mean)
Merb Edge 913: Requests per second: 723.98 [#/sec] (mean)
Rails Edge 8117: Requests per second: 375.47 [#/sec] (mean)

10 Partials
Merb 0.4.0: Requests per second: 150.57 [#/sec] (mean)
Merb Edge 913: Requests per second: 424.83 [#/sec] (mean)
Rails 1.2.5: Requests per second: 214.91 [#/sec] (mean)
Rails Edge 8117: Requests per second: 213.56 [#/sec] (mean)

100 Partials:
Merb 0.4.0 Requests per second: 22.42 [#/sec] (mean)
Merb Edge 913: Requests per second: 100.83 [#/sec] (mean)
Rails 1.2.5: Requests per second: 45.15 [#/sec] (mean)
Rails Edge 8117: Requests per second: 44.23 [#/sec] (mean)

* Merb edge without view context caching on partials
* Experimental ONLY

0 Partials: Requests per second: 727.22 [#/sec] (mean)
10 partials: Requests per second: 573.50 [#/sec] (mean)
100 partials: Requests per second: 229.13 [#/sec] (mean)

The setup - Rails

rails partial_speed_test
cd partial_speed_test
ruby script/generate controller tester

# setup AR to point to an emtpy db

# setup your controller See Below
# setup routes

mongrel_rails start -e production
# run the ab tests

The Setup - Merb

merb -g partial_speed_test_merb
cd partial_speed_test_merb
ruby script/generate controller tester

#setup your controller See Below
# Setup routes

merb -e production

Controller setup

# app/controllers/tester.rb

def index
render
end

# app/views/tester/index.html.erb (RAILS)
Inside Index
<% 10.times do %>
<%= render :partial => "blah" %>
<% end %>

# app/views/tester/index.html.erb (MERB)
Inside Index
<% 10.times do %>
<%= partial :blah %>
<% end %>

# app/views/tester/_blah.html.erb
BLAH<br/>

Merb Tests

Procedure

  1. Uninstall merb gem

  2. Install merb gem

  3. Start app

  4. Start tests


Released 0.4.0 gem
svn co with revision 913 and rake install
change the partial method to use cached_view_context instead of clean_view_context(engine) in render.rb

Rails Tests


  1. Generate an app using the 1.2.5 gem and start it up

  2. Run ab tests

  3. rake rails:freeze:edge (8117)

  4. rake rails:update

  5. Start app and run ab tests

The number of partials was changed by changing the number in index.html.erb

Phew.. That was a long post.

Just to reiterate, take this with a grain of salt. This was just a test for my curiosity and I encourage you to run these yourself if you are so inclined. Hopefully I've captured all of the setup.

Update: Again with these numbers take them with a grain of salt. These numbers are from the activity monitor on OS X
Rails 8117          MEM(MB)         CPU       
10 Partials 24.9 99.7
100 Partials 25.8 100.0

Merb 0.4.0 MEM(MB) CPU
10 Partials 15.6 98.8
100 Partials 21.4 100.0

Merb 913
10 Partials 14.3 98.8
100 Partials 13.8 100.0

Tuesday, November 6, 2007

Merb Generators Now Read Your Mind

Three days ago I blogged about the new generators in merb. Meh, that's so 3 days ago ;)

Update: If you've installed these gems from trunk since 0.3.7 and before the official release of 0.4.0 you should first uninstall the merb, merb_active_record, merb_data_mapper and merb_sequel gems before you install them again.

Update #2:use_orm :active_record and :data_mapper have been changed to :activerecord and :datamapper respectively.

ivey and wycats, regulars on #merb and all round good guys today rolled up their sleeves and removed the ugly contstant Merb::GENERATOR_SCOPE that you had to modify to get the generators working. They also came up with a way to implement default model and resource_controller generators in merb.

How to use it now?

Now when you setup your brand new merb app, you go into config/dependencies.rb as usual but instead of
dependency "merb_data_mapper"
you use
use_orm :data_mapper
# and
use_test :rspec
This will setup the dependency and also sort out what generators your app will use. You've got a good selection of generators now.

  • migration - a migration file

  • model - a model in the correct format + migration + tests / specs

  • controller - a blank controller + tests / specs

  • resource_controller - a controller setup with basic CRUD actions + tests / specs

  • resource - model and resource_controller and all things associated with them

So the one you'd normall use is resource.
ruby script/generate resource my_resource attr:type attr:type
This will generate a model and resource controller complete with tests / specs and migrations in the syntax of your selected ORM.

You can also use all of these generators even if you don't use an ORM at all. In that case it will just give you plain vanilla ruby classes with attr_accessors

Neat. These generators are very new though and I'm sure the content of the generated files can be improved. If you find something that you think needs to be changed please raise a ticket preferably with a patch.

wycats blog has a great outline of the major new features of the imminent version 0.4.0 of merb. If your interested you should check it out.

Saturday, November 3, 2007

The New Merb Generators

Update: The general information in this article is still correct, but the syntax is now nicer. You can read about the new syntax here.

In preparation for the big 0.4 release of Merb, the generators have had a major overhaul. Merb is ORM agnostic and uses plugins to harness an ORM. There are currently plugins for ActiveRecord, Sequel, and DataMapper.

The ORM agnostic goal of Merb, is a great thing, developers can use what they're comfortable with, or what suits a particular task at hand. This does create it's own issues of how to support it in the framework.

There's been great work done on merb to make this reasonably painless on selecting which ORM to use. Once the gem for the plugin is installed, a quick trip to config/dependencies.rb, and uncomment the line for your plugin and your away. But with generators it's been a bit more difficult up to now.

You'd think that will the option of ORM, and Rspec vs Test::Unit things could get a bit murky. Up to now the generators have been named specifically for the ORM you want to use so you can get the right syntax and parent classes etc. The only controller generator was just a blank stub that you have to fill out each time. Well now there's a new way to manage your generation needs.

Rubigen is awsome

DrNic extracted the generators from rails and added some of his magic to them. The result is a very clever and flexible system for generating files for your app and Merb tries it's hardest to put all this goodness to work for you.

Ok Already. Example Time

After you've installed the gem for your ORM plugin you need to tell Merb to use it for generators.
# config/merb_init.rb

# Put your selection for generators in here. I'll use DataMapper because I like it :)
Merb::GENERATOR_SCOPE = [:data_mapper, :merb, :rspec]
This will tell Merb to generate with Datamapper, and Rspec. This could have been :active_record and :test_unit if you want to go down that road. The order you put these in will matter in future (hopefully ;))

Generate A Model

ruby script/generate model my_model attr:type attr:type
This will generate the correct model, migration if required, and the correct test or spec stub. If you change the GENERATOR_SCOPE to use :active_record and generate another model, you'll see that the correct files are generated.
Note: This currently is not available when you're not using an ORM plugin. There are plans to allow this but at the moment it's not happening.

Generate Controllers

There are two ways to generate a controller.
  1. merb_controller
  2. resource_controller
The first, merb_controller just generates a blank controller stub, helper and tests or specs. This is available all the time, even when you're not using an ORM.

The other is available when you use an ORM plugin.
ruby script/generate resource_controller my_model
This generates a CRUD controller with the correct sytnax for your ORM selection you specified in merb_init.rb along with the helpers and test or spec stubs.

Currently there is no "resource" generator that takes care of a model and controller and all the associated files. This is on the drawing board and hopefully it won't take too long to get it sorted.

Change the ORM, mix and match your test_unit or rspec selections to see it in action. I wouldn't experiment by changing scopes in a real project though since Merb currently only supports using one ORM at a time.

Friday, November 2, 2007

Merb Crud Controller

The other day I decided to try a complete CRUD controller using the new provides format and Active Record. It supports XML, Javascript, YAML formats where appropriate and HTML across all actions. The built in error handling in Merb also helps to clean things up.

Well. Here it is, for better or worse. I really like it though ;)

class Posts < Application
provides :xml, :js, :yaml

def index
@posts = Post.find(:all)
render @posts
end

def show
@post = Post.find(params[:id])
render @post
end

def new
only_provides :html
@post = Post.new(params[:post])
render
end

def create
@post = Post.new(params[:post])
if @post.save
redirect url(:post, @post)
else
render :action => :new
end
end

def edit
only_provides :html
@post = Post.find(params[:id])
render
end

def update
@post = Post.find(params[:id])
if @post.update_attributes(params[:post])
redirect url(:post, @post)
else
raise BadRequest
end
end

def destroy
@post = Post.find(params[:id])
if @post.destroy
redirect url(:posts)
else
raise BadRequest
end
end
end