The Freak Parade

I did a pretty exhaustive search for an ESB recently for use on a project that we’re getting underway. FYI, my selection criteria was coming from the point of view of an SMB, so we probably emphasized simplicity and ease of use over scalability and robust governance tools, for instance. Also, cost was a big factor, so the big infrastructure vendors weren’t even on the radar, even though I’m sure they could be used quite easily in a .NET environment.

Anyway, there weren’t all that many options. I guess I wasn’t surprised, but I was a little disappointed. The Java community has half a dozen or more full blown open source ESB projects, many with professional support organizations where you can buy support and consulting if you need it.

In .NET land there wasn’t anything like that. Here is what I found:

BizTalk Server + BizTalk ESB Guidance = ESB ?

Initially I assumed we’d be going down this path, so I read a really great book by Darren Jefford on the subject. BizTalk is a pretty deep piece of software, but like a lot of Microsoft frameworks it aims to be all things to all people, and we wouldn’t need 75% of what it provides and it didn’t offer the simplicity of development, maintenance and deployment I was looking for. The business rules chapter of the book opened up with a warning that the BRE in BizTalk is complicated beyond measure, yet you should learn it because it is powerful. To make it an interpretation of an ESB you apply the ESB guidance put out by Microsoft that I have heard eventually was the jumping off point for Neuron (possibly I made this up in my head), but the guidance exists and seemed to have quite a few moving parts. The installation instructions and prerequisites weren’t a walk in the park, and the whole itinerary based approach isn’t quite what we needed. We were looking for dead simple Send, Receive, Publish, Subscribe semantics in a code based API. BizTalk is a lot of things, but not those things, so the search continued.

Neudesic’s Neuron ESB

Next up we evaluated Neuron. This is really a pretty fantastic product that combines the easy API I was looking for with all the powerful translation, BAM and orchestration of BizTalk (but you need to license both products, I don’t believe Neuron offers these features separately) with a ton of flexibility and some great centralized endpoint configuration mechanisms. From what I gather they pretty much pulled BizTalk apart and exposed the functionality of its pieces (orchestrations, BAM, transformations, adapters, etc) directly via their messaging API and bypassed the MessageBox entirely, which is a pretty slick trick.  Without BizTalk, I think Neuron still offers an alternative to orchestrations using integration points with WF and like BizTalk has a pretty good set of legacy systems adapters. Still, even though Neuron is orders of magnitude less expensive than the big dollar enterprise ESB’s, it was still more than we wanted to spend, mostly because our messaging needs were so simple.

Mass Transit

This is a pretty new open source project, currently at version .1. The code is really pretty interesting, there seem to be plenty of extensibility points and the test coverage is very good. Lot’s of moving parts in this one too, but it is a rapidly evolving project. I’ve been keeping an eye on the code as it moves from .1 to .2 and the API is changing rapidly. I think this is definitely worth keeping an eye on, but as of this exact moment in time the API is a bit too volatile, as you would expect in a .1 version. Once it cooks for a while and sees some use I think it will be a nice alternative to the others on this page. Also, they have an NMS transport for integrating with ActiveMQ, although that may not be ready yet.

nServiceBus

This is a relatively mature open source project currently at version 1.8 (RC2 or something) and from accounts has been used in some pretty demanding, large scale production applications. It is a one man show, from what I can tell, run by Udi Dahan, who has given himself the moniker of “The Software Simplist” and I think that shows in his service bus. nServiceBus has the least number of moving parts and it is very easy to understand the entire code base without furrowing your brow. It is trivial to set up endpoints and get them sending, receiving, publishing and subscribing. This is the library we are tentatively using in our project. I wish, at times, it had a few more extensibility points, or were a little more modular, as it comes apart in roughly three or four big pieces. You can replace any of the big pieces, but there isn’t a whole lot of customization or modularization within the pieces. My belief is that Udi Dahan vigorously defends the simplicity of his project and avoids adding what he considers complexity for the sake of extensibility he doesn’t see the value of. By and large I would say nServiceBus codifies a particular approach to service buses and SOA in general and is deliberately designed to enforce that philosophy. Actually, I think the website says something to that effect too. One nice thing about nServiceBus is that Udi Dahan has lots of blog posts and pod casts discussing in articulate detail his SOA philosophy, which if you agree with it will you find direct support for those ideas in the project. The thing comes with a MSMQ transport, but the contrib project has an ActiveMQ transport and it sounds like an HTTP Transport will arrive shortly with a WCF/MSMQ transport coming at some indeterminate time after that. [Update] We’re now using Simple Service Bus, our own derivative of NSB, description below

Simple Service Bus

Simple Service Bus is a derivative (fork) of nServiceBus. Full details on the project can be found here, but the idea was to take the reliability and simple API found in nServiceBus and add in the extensibility and modularity required to enable us to design messaging systems in a more “building block” way, for better or worse. A pdf document providing a comprehensive overview of the project, including an architectural description / diagram, can be downloaded here. The project home page on CodePlex is here.

BizTalk Services

Microsoft is building a SaaS messaging bus with some slick features, orchestrations via WF and more significantly, I think, some sort of federated identity service using (I believe) a standards based STS service. However, it is currently in CTP and is not at all ready for production use (according tot he website). Plus, nobody knows what it will cost.  I think it is designed for partner-partner integrations and I’m not sure if it is intended for the Internet Service Bus (ISB, what they are calling BizTalk services) to be a replacement for an Enterprise Service Bus, but perhaps a compliment. Not sure on that one though. In any case, it is worth watching (and possibly playing with if you have time) but not an option for production use just yet.

Linxter ISB

Another ISB in beta, this service is similar ( I think ) to the core purpose of BizTalk services (hosted, secure messaging across the open Internet), with a slightly different feature set. I posted about Linxter here.

That
’s all I could find. Personally, I think there is room for more. Resources below. Links to project/product home pages are in the titles of the products themselves.

Neuron-ESB-vs.-Microsoft-ESB-Guidance-A-Quick-Comparison

Neuron posts from Sam Gentile

NServiceBus – Makes Building Enterprise .NET Systems Easier

http://udidahan.weblogs.us/

Mass Transit author blog

Very brief review of Mass Transit code

Simple Service Bus – Overview & Architectural Description

First, I should say I am a total neophyte when it comes to SOA or ESBs. I just stumbled into this SOA party, with my fanny pack and my walking stick and my perpetually bewildered expression. But as a neophyte struggling to understand concretely what I instinctively understand is a vitally important concept or movement or architectural style or religion or whatever SOA is, I must say my bewildered expression isn’t out of place. Many blog posts or articles on the subject open with a comment on how notoriously undefinable SOA seems to be – at least to the extent that experts in the field, whoever they are, can agree on something. There are some things everyone seems to agree on, though. The most prominent of these is “SOA is not a technology.” This is often followed with a stern warning about how you can’t “buy it”, it doesn’t come “in a box”. There also seems to be consensus among those who do a lot of thinking (or at least writing) about SOA that it is about “Business-IT alignment.” That sounds like a catch phrase to me that doesn’t communicate a whole lot of useful information, but I take it to mean that the practice of SOA is a holistic practice, business wise one that seeks to understand the larger strategic goals of the organization as a whole and organize and coordinate its myriad software systems and the business processes they support in service of the Big Picture, the Big Picture ultimately and inevitably being profitability (what else is there in this glittering capitalist wonderland of ours?)

And then there is the ubiquitous prejudice (bordering on hatred) that SOA and its practitioners seem to universally hold against the poor beleaguered “Silo”. (Oh Silo, who weeps for thee?)  And finally, to finish up the things everyone seems to agree on, it seems that SOA is definitely not JBOWS.

Oh, and SOA is loosely coupled. It is definitely loosely coupled.

However, none of those aspects of SOA says much about services or architecture, except maybe the loosely coupled part. Business alignment, agility, “not technology” – those seem like outcomes or goals of a particular type of architecture rather than a description of an architecture – or even a description of an architectural style (like the mud hut or Victorian mansion). Nevertheless, SOA is about architecture, because architecture is in its name – but it is the concrete part of the thing that seems to be in such hot contention – do you need WS-*, an ESB, must you have a regipository, should messaging be asynchronous, Request Response, point to point, brokered, RESTful, Queued, should services be layered, data oriented, process oriented oh goodness, just don’t buy it in a box! Because that WON’T be it!.

Well, it is all very confusing. You kind of have to pick a camp and do your designs that way, when you’re fresh meat like me and haven’t been seasoned in the fires. I myself have selected from the shelves a nice, fresh Bus based, business unit centric sort of approach to SOA a la Udi Dahan, but am still very curious about the others, the ones with repositories and registries and that use WCF and WS-* and REST and dynamic routers and whatnot.

Anyway I was reading Sam Gentile’s educational SOA series  and it led me to reading some other thingsand I came across a metaphor for SOA that finally made it click for me in a real way, and now I’m not so perplexed about all the various hotly debated approaches because they are just (very important but equally valid) different approaches to a unified, relatively stable vision of what SOA is – because SOA is technology, of course, and it is architecture, but it isn’t a style of architecture, like mud huts VS Victorian mansions, it is a scope of architecture, like Building Design VS Urban Planning. SOA is about turning ad hoc communities of software and process into an integrated economy composed of towns that are part of larger counties that are part of larger states, and so on. SOA is about the design and execution of the master plan, the infrastructure and government and laws that all of an organizations IT entities must follow to enable peaceful, productive commerce all around. And what isn’t mandated by the law of the land is left up to the organizational units to rule themselves and their component sub communities as they please. How autonomous of them. All very hierarchical and organized. I also suspect (based on what I’ve read, mind you, not what I’ve experienced) that SOA’s often tend to grow like many cities, patches of disorganized systems agreeing to abide by a common set of rules, setting up transit systems and infrastructure between them, and these become cities which become models for other areas of the organization to follow suit.

What I like about this metaphor is that it captures many of the individual desired outcomes of SOA pretty nicely in your brain, and this is important because trying to picture what “driving increased strategic business agility and alignment into the enterprise” looks like in a way that can help you build it is a rather tall order.

For instance, perhaps each LOB application is something like an individual business or even a person who has some autonomy, makes some decisions for himself, but for the most part he must communicate in a language common to his local community and must follow the local customs, such as a specific technology stack. The larger communities to which each citizen belongs have more autonomy as a group. Maybe these are Services. They have their own culture, their own customs, and by and large they do things their way. But as a unit they must participate in the larger economy, they must import and export goods and services to other communities in an organized, coordinated way and report progress and pay taxes to the governing body and so on. Therefore at least those members of each community (endpoints) charged with conducting business on behalf of the community must,  in addition to understanding local culture and customs, also speak the national language, convert local money into a common currency, understand the global business protocols.

Clearly, you can’t “buy” something like this – you don’t march into to the wild west and order your “city” from the Sears catalog (or your favorite vendor), complete with a set of laws and a functioning government. You can buy a bunch of concrete and tools (such as WCF), or even a fully functioning subway system (such as an ESB), but the thing itself, the transformation of wild tribal cliques into a coordinated, modern, centrally governed “society” of connected systems, must be built piece by piece, tailor made, using many different kinds of physical infrastructure along the way as appropriate while implementing appropriate rules, laws, protocols, etc. For the whole rogue crowd to work together in a single, functioning digital economy you really need to gather the tribal chiefs and understand their needs and get participation and buy in because if you don’t then if not an outright revolution at least a certain amount of subversive activity is certain to undermine the functioning of your new society. No vendor can sell the solution to that.

Incidentally and off topic, this paints me a nice image for JBOWS. I like to imagine taking disparate groups of people over a wide geographical area who all speak different languages and dialects and have totally different cultures and giving each of them a telephone and a set of phrase books and expecting them conduct meaningful and efficient commerce. Throw a phone book in the mix and you have JB
OGS.

So anyway, this may or may not resemble what people mean when they talk about SOA, but it is what I have managed to glean, academically speaking, from all the shrapnel that is flying around.