The Freak Parade

[UPDATE] A pdf document providing a comprehensive overview of the project is now available here.[/Update]

I have posted a new project on CodePlex, Simple Service Bus. Simple Service Bus is a derivative of Udi Dahan’s nServiceBus, which I’ve mentioned before. I’ll be posting some more extended samples and documentation for using the new library, but first I wanted to document my motivations. nServiceBus is a very solid, very useful, very easy to use piece of software for building asynchronous, messaging based service endpoints, and it’s creator is a well respected SOA guru, at least in .NET circles. I, of course, have no such pedigree, which is why I chose to start with nServiceBus as a foundation rather than starting from scratch.

So what is wrong with nServiceBus that required us to fork it? Well, nothing. My take on nServiceBus and the direction Udi Dahan seems to take the project is that it is a library designed to provide just enough rope to get the job done, but not enough rope for us developers or architects to hang ourselves from a scalability and reliability perspective as we wade into the asynchronous, messaging based waters for the first time. This is a very useful trait for a library to have. However, such an approach comes with some tradeoffs, namely when it comes to extensibility and customization, and I’m a lego brick kind of guy, maybe stemming from control issues, who knows
In service, I believe, of the protective and prescriptive nature of the project, nServiceBus has many of its most interesting parts “soldered to the motherboard” - their implementation is blended in with the default implementation of the framework itself, which means that if you want to customize how certain things work, you have to diverge from the codebase. Now, to be fair, this is only true to a certain extent. nServiceBus has several major pieces that can be swapped out, such as IBus and ITransport, but within the responsibilities of the default implementations of IBus or ITransport are a number of smaller components that can’t easily be customized.

It hardly seems necessary to go so far as to fork the project when custom IBus and ITransport implementations could have accomplished 95% of the flexibility I was going for. And, in reality, it probably wasn’t necessary - but once the thing was already on the operating table, I couldn’t resist making some preference based changes to the IBus API, and I wanted to add custom headers to the message envelope (which crosses between IBus and ITransport) and at that point there really wasn’t any turning back.

Another question would be, well, if nServiceBus didn’t have the extensibility you were after, why not choose Mass Transit? It is extensible. And, in fact, it is - but it is still fairly young, and we’ll be going into production very soon. nServiceBus has been through the fires a few times, and I figured I could add the hooks I needed without altering the fundamental design of how nServiceBus operates, which I quite like and I’ll sleep better at night because of it. My belief, though, that if it was six or eight months from now, Mass Transit would have been the direction I’d have taken. Also, I already had a project, NsbExtensions, that implemented endpoint health & performance monitoring features on top of nServiceBus, and I wanted to be able to reuse all that code.

Here are the design goals of Simple Service Bus - most but not all of which are realized but not thoroughly tested in the version currently posted to CodePlex:

* Small DLL footprint - nServiceBus (for valid architectural reasons) is split up into eleven thousand, two hundred and nine different DLL’s. That may be an exaggeration, but many of the dll’s have only one or two code files in them, with only a few lines of code each. Simple Service Bus comes in three DLL’s for a normal endpoint, and add’s a fourth to implement an Endpoint Health Monitoring Server. Also, required third party dependencies are kept to a minimum (0).

* Custom headers in the Message Envelope - I wanted to be able to attach bits of information that are necessary for the messaging system to function properly, like authentication tokens or endpoint ID’s, without having to include those pieces of irrelevant data on the otherwise domain-oriented messages themselves. This results in a somewhat cleaner set of message contracts, but it also allows you to do some content based routing or authentication checks on a message before the payload ever has to waste the CPU’s time on deserialization. If a message is going to be blocked due to lack of authorization, or re-directed to a different endpoint, or simply written to disk as part of an audit process, there is no need to send the payload through the de-serialization / messaging handling process.

* Send and Receive Pipeline Pattern for Message Processing - instead of having a hard-coded path from Transport through final Handling, Simple Service Bus provides a configurable Pipeline approach, where each interesting step in the messaging processing process is implemented as a pipeline component. This allows programmers to add, for example, a content based message router ahead of the payload deserialization component, or to add a statistics gathering component at various places within the pipeline. Additionally, a rich set of events are fired while a pipeline is processing a message, allowing fine grained performance monitoring or logging if required.

* Replaceable Subsystems - each major concern of the bus has been factored into a simple subsystem as defined by an interface. The default implementation of any subsystem can easily be replaced. For example there is a Message Dispatcher subsystem, a Subscription Manager subsystem, a Message Serialization subsystem, a Message Type to Message Handler Resolution subsystem…etc. In this way, if you want to implement a Subscription Manager that can evaluate publisher-side expressions to determine whether or not to send a message to a particular subscriber using a custom DSL or any other expression evaluator, you can easily swap out the existing subscription manager with your enhanced one. And you can do it for just one endpoint, if only one endpoint needs that special functionality.

* IoC supported, but not required. The building of objects is handled by subsystem as well, and the default implementation can be given a reference to an IoC container to build all objects and services, or it can use simple reflection if no IoC is provided. To accommodate this, a Service Locator was used rather than a pure dependency injection approach, but the services registered with the Service Locator can optionally be built with an IoC. Also, of course, all Message Handlers and Pipeline Components will be constructed on demand using an IoC if provided, or simple reflection otherwise.

* Polymorphic Message Handlers - the default Message Handler Resolution Service allows message handlers to be defined for interfaces or base classes, allowing the creation of message handlers for families or categories of messages rather than just individual messages. This also applies to subscriptions, meaning that an endpoint can, for instance, subscribe to IEventMessage, and receive all messages that implement that interface, instead of being required to subscribe to each concrete message type.

* Endpoint Health & Performance Monitoring built into the core library - all endpoints can be configured to provide status reports to a monitor service with a few lines in the configuration file.

* Simplified Configuration - this is debatable, because Udi Dahan has gone to significant lengths to partition “programmer configuration” from “administrator configuration” - the former being accomplished in code and the latter being accomplished in a configuration file. This is probably a wise, experienced based approach, but I myself like the option to configure everything in a single, simple block of XML, if XML is being used. So that is how Simple Service Bus does configuration, in a more traditional manner.

* Abstract base classes provided for easily creating custom Transports, Pipeline Components, Performance Probes, etc., abstracting much of the ugly details and allowing the system to be extended by overriding a few simple, targeted methods.

* Fully Unit Tested - this isn’t finished yet (I’m still trying to get the hang of TDD, so the tests weren’t written first in this case) but the idea is to have unit tests for each piece of the system, allowing modifications or customizations to be done in a safe, controlled manner.

*object based API. I removed the IMessage marker interface for Simple Service Bus because (perhaps mostly for aesthetic reasons) I wanted our Message Definition dll’s to be as clean and reference free as possible. So now Send, Publish, etc. operate on System.Object rather than IMessage.

And that’s about it. I fell in love with Gregor Hohpe’s “Enterprise Integration Patterns” and wanted to apply those patterns to my own evolving messaging system, so this effort is sort of centered around enabling nServiceBus to accommodate that desire in a clean, plug and play way. We’ll see how it goes.