TG’s Killer Features: Object Dispatch

When people ask me what really sets TurboGears apart from the rest of the frameworks out there, I throw away everything we have built upon our foundation and find one thing remaining.  Object Dispatch.  Dispatch is probably a heinously boring topic for some folks.  For me, it’s a fascinating topic, one full of complexity, twists, and turns.  This is my main contribution to TurboGears, creating a dispatch system which is flexible enough to handle the most complex url mapping you can come up with, simple enough for the average Joe to use.  Like most of the technologies TG builds upon, the dispatch system is designed to be easy to start with, and easy to mould when you find that the defaults don’t meet your needs.

Let’s look at TG’s humble roots.  CherryPy was probably not the first framework to realize that nested classes are a fantastic way to organize urls.  Afterall, web addresses are nested, right?  (More on that later)  As the web server of choice for our founder, Kevin Dangoor, CherryPy offered a solid foundation in what remains in my mind to be the killer feature of TG.  Without OD, you pretty much don’t have TurboGears.  (You have Pylons).  Speaking of our brethren, lets look at a few frameworks to see how you dispatch a simple url.

Let’s take this url for instance:

rules/section/3

Pylons:

in routes.py you would add the line:

map.connect(’/rules/section/:id’, controller=‘section’ action=‘show’)

create the controller:

class SectionController:

def show(self, id):

return ‘i am a section, hear me roar’

Django:

edit urls.py to add:

(r’^rules/section/(?P<id>\d+)$’, ‘myapp.views.section’),

create the controller:

def section(request, id):

return HttpResponse(‘i am a section, hear me roar’)

Both are remarkably similar here, but I would give the advantage to Django for two reasons.  First, it uses regular expressions.  That means I don’t have to learn yet another dialect for matching strings.  Second, it specifies the controller directly, instead of the framework determining the location of the controller code with some magic lookup method.  Lastly, it provides a serialized method for routing, which means that you can provide a set of lookups in a file which can be digitized, modified, and re-serialized.  This is a huge advantage if you are planning on making a large system of swappable parts.   However, I don’t think the routes system is bad, and I have built my own system around it to manage the above problems, for a very plugin-required system I am building.  By the way, I’m no Django expert, so please feel free to comment on my code samples and I will fix them if need be.

Okay, so let’s look at the TG version of the same thing.

routing match code:

huh?  what’s routing?

controller code:

class SectionController:

@expose()

def index(self, id):

return ‘i am a section, hear me roar’

class RulesController:

section = SectionController()

class RootController:

rules = RulesController()

Hmm.  Notice the lack of weird symbols in the router code?  Oh, waitaminute, there _is_ no routing code!  That’s one less thing for someone to learn, one less thing to have to code.  Except there’s a catch.  That little @expose; what’s that all about?  Well, because the routing is done with introspection, you have to tell the dispatcher that something is _allowed_ to be dispatched upon or not.  So, already you have addressed some security concerns, where otherwise you would have to test that your routes don’t end up in a method that should not be exposed to the outside world.  And in complicated routing, this can be a definite issue.

So, Object Dispatch provides a provably simpler path to nirvana for the web programmer, but it’s just not cutting it for you.  Well, first off, with TG2 you can easily drop back and use Pylon’s routing mechanism.  Secondly, we build a RestController, which routes based on HTTP Verbs.  Lastly, in TG 2.1 you have the cream-de-la-creme of dispatch power.  You may write your own dispatch mechanism.  Theoritically, you could write a DjangoDispatchedController, that uses regular expression to determine the enclosed members’ dispatch mechanism.

Wait, huh?  Okay, so dispatch works in a similar manner for every framework, but not every framework can so easily switch dispatch mechanisms on the fly.  Here’s basically how it works in TG:

First, we go through the normal Pylons routing, this usually deposits us at the RootController of your app, unless you have overridden the pylons routes for your app to do something else.  Next, the routing mech takes your url, and splits it by ‘/’ into a list of strings.  It now looks for attributes of the root controller that match the first string in the list.  If that attribute is a method, and the method’s number of arguments jives with the remaining items in the list, the dispatcher says it’s done, and fires that method.

If the attribute found in the initial dispatch is an instance of a class, and the class has no _dispatch method, it traverses that class with the remaining elements of the list, repeating down the list until it finds a method that matches.

Now, if the attribute is an instance, and it has a _dispatch method, the dispatcher will then _become_ the _dispatch method, and dispatch will continue in whatever method was identified by the new _dispatch method.

So, if you read that whole description, I commend you, but if you only looked at the picture that works too.  Basically what I am saying here is that TG2 provides you with the ability to create your own method for dispatching your controller code based on the remaining URL in the path (that’s the “black” box).  And it works too, this is how we implemented TGController and RestController, and soon we will have an AmfController that does a Remote Procedure Call method of dispatch, to help simplify controller code for our Adobe Flex friends.

TurboGears is in my mind designed correctly, with very little tradeoffs made for the sake of pragmatics.  This no-sacrifices approach to the framework is yielding results.  As a developer, choosing TurboGears means that you can get started pretty easily, and are not stuck once you hit the limitations of TG’s developer’s imaginations.

I have decided to make this blog entry into a series on what makes TG 2.x a compelling choice as a web framework.  The next blog entry will be:  TG’s Killer Features: SQLAlchemy (obvious, no?)

Tags: , ,

4 Responses to “TG’s Killer Features: Object Dispatch”

  1. Jorge Vargas says:

    You forgot one little detail. The 2.1 dispatch is much more feature full than 2.0 and it’s faster.

    Also didn’t mention the WSGIAppController and the CrudController/AdminController those are awesome :)

  2. This sounds a little like the object graph traversal based dispatch that Zope has had for a decade or so. repoze.bfg has a more generalised version of this too.

    I would argue that implicitly matching attributes and methods can be confusing and possibly lead to security issues (did you really mean to let people traverse to the _get_plans_for_world_domination() helper method) if you’re not careful. Zope 2 is sometimes prone to that, unfortunately. I’m sure TG has a solution for that, though.

    The view in Zope land seems to be that the most useful API for object based traversal is __getitem__. Traversal to / gives a root object. /foo calls root['foo']. /foo/bar tries root['foo']['bar']. Obviously, a custom __getitem__ can be used for more advanced matching.

    Of course, we’re not quite talking apples and apples here, since in Zope you’re really traversing an object graph of model objects and looking up views on that model object once you found the one you want to publish. But the theory is similar, and repoze.bfg generalises it a bit further, supporting both model based traversal and something similar to routes.

    Martin

  3. percious says:

    @Martin

    Actually, I addressed the security issues in paragraph 4, describing how the @expose decorator works.

    I have had quite a few discussions with Chris McDonough on how repoze.bfg is doing dispatch. I feel like the Zope model traversal is the right solution in some cases, but that it often mixes the M with the V in MVC.

    The idea of using a custom __getitem__ is a good one, but the question is whether or not the “remainder” of the url hence parsed can be aquired from within the getitem method. Also of use to the developer in TG2.1, is a list of controllers you visit along the way is accumulated as you traverse, so that it is easy to see the path you have taken to get to your current state of dispair, ehem, I mean dispatch.

    The fact is, debugging dispatch code is challenging, due to it’s recursive nature. TG is providing the developer with not only the power to provide their own dispatch, but the tools with which to debug the state of the lookup at any time.

    There’s actually quite a few things I left off here on TG’s dispatch. If people so desire I can write a deeper look into the dispatch mechanism.

    @elpargo

    Yes, Crud/Admin/WSGIApp controller are all compelling parts of TG2, but they do not have a dispatch component, due to the fact that they are in fact based on TGController and RestController.

    And yes, 2.1 dispatch is faster than 2.0. This is especially true when using embedded RestControllers. The amount that TG re-evaluates state of dispatch is much less. I also think I can make routing EVEN FASTER in TG by leveraging Pylons’ routes and caching known paths. Since the Controller objects are embedded one could also traverse the _simple_ parts of the tree ahead of time and determine a cache at startup. If only I hadn’t a day job….

    cheers.
    -chris

  4. Chris Arndt says:

    If I understand correctly, one important difference between the lookup mechanism TG2 uses and CherryPy’s mechanism used in TG1 is that CP tries to match the URL to the longest object trail it can find and then backs up the object tree until it finds a matching method, whereas TG2 uses the first method it finds searching from the the top of the object tree. You can read about CP’s approach in the CP tutorial: http://www.cherrypy.org/wiki/CherryPyTutorial

    Is my interpretation correct and, if yes, why this difference? Was this a conscious design decision?

Leave a Reply