07/24/08 :: [SOA] Here we go again [permalink]
There are some touchpoints between the REST and the ESB debate. It seems that we humans love to be for or against something. It does not matter if a bunch of people are happy going about their business, somebody has to come in and tell them that what they are doing is either bad, stupid or immoral. I am a reluctant pro-choice guy (how can you be against life?) but you can't stop thinking that if all these people could get together and spend their energy to give the tools to people be it spiritual or physical to avoid unwanted pregnancies, you would not have to be pro or against anything. It is of course a lot easier to be pro or against something.
I am wondering what part of our evolution lead us to become so polarized. I don't know if you read Judith Harris's book Nature vs Nurture. She explains that she believes intelligence emerged as groups of humans had to always split when they reached a critical size. She believes that each member of the original group would then go to the group he or she had the most affinity with. This constant split of population was always along some characteristics such as, the smarter people would get together. I don't know if I am doing justice to her arguments but the book is a good read, specially if you are a parent.
Please note that I would leave the (other) REST alone if they somehow would leave us alone too, but no, that would be too easy. Again, the only reason for me ever restarting this debate last fall was that Tim Bray's crap on "the end of SOA" ended in my inbox and was circulating in the company where I work creating FUD for our SOA effort. So I took on the mission to understand and the more I understood, the more I realized this was a hoax. Be assured that by now, Steve, Stefan, Tim, Stu, Subbu, Pete and many others would have written scores of articles detailing how practically (and pragmatically) you could replace WS-* with the (other) REST. The only reason they don't (and instead keep writing these insipid baby formula articles) is simply because they can't.
So let's go back to the ESB debate. I don't expect I will be able to put an end to it (it really does not matter) but hopefully a few readers will be able to use this information. I don't know if you recall the post I wrote on "REST creates a strong coupling". Well I was right, when Stefan finally admitted how he was using a mixture of message passing and CRUDing, in a contract-less fashion to communicate between a resource representation consumer and a resource representation provider, then it became clear that when a human is not in the loop REST creates quite a terrible coupling.
How does it relate to ESB? Well SOA is about creating autonomous assets which are flexible enough to be composed and participate in multiple solutions. In other words, here I am with two pieces of code that were written by different people, at different times, in different technologies, that scale differently, have a different security model... The question one would naturally have is "how can I make them work together to achieve a particular goal (often unforeseen by the designers of these pieces of code)?"
Here are my 4 tenets of loose-coupling.
- an asset must be autonomous
- an asset must be modular (i.e. composable)
- an asset must be able to evolve (to a certain degree) without breaking existing relationships
- an asset must be capable of a certain degree of variability to accommodate different consumers
We all understand how traditional remoting technologies fit in that picture, well they don't.
First let me say that if someone were to rename ESB as "Service Container" we would all go in peace and claim "yeah" I need a service container, who doesn't. But that would be way too easy. The question becomes what kind of Service Containers do you need? Is there only one type? Do you need a service container at all?
Now, let's assume I have a legacy system running an App that I really like, I invested millions in writing (or buying, or customizing) this code. It was written years ago in an unsupported technology but it is running. I have seen recently an application that was launched in 1974 and was still running. Yeah, not every piece of code was written with REST or WS-* in mind. So the question remains, how can I create a service be it RESTful or SOAP-based from this kind of code?
Well there are not that many solutions to achieve loose coupling.
The picture below represents a provider and a consumer. They are both peers, the consumer is the one who initiates the interaction. They have both been implemented at a different time (a mash up is a typical example). Because they have both been developed at a different time, they both assume an "internal" representation of the other side. This is their view of the interaction. Ci is the consumer internal interface and Pi the provider's one.
Since they are developed independently we can never assume that Pi = Ci (yet this what remoting technology do all the time, this is why remoting IS-A failure). So what do we do? As Paul Brown would say, you need a level of indirection. Yeah, there is no magic wand that makes loose-coupling serendipitously happen. We introduce a second set of interfaces: the external ones: Ce and Pe. These interfaces are isomorphic and are defined to work together, at t=0, Ce = Pe.
"External interfaces" are different from "internal interfaces" because consumers may use a different security model, business protocol, scalability, interoperability model, transport... So in the end you have to write some glue code (LCc and LCp).
The question becomes what's the best way to write this code? could a service container help me write this code? from a topology perspective, is it worthwhile to consider using the same service container for writing both LCc and LCp? or should they be separate containers? Well, there is not a perfect answers. Provider and Consumer are logically autonomous, but there is no reason why they can't coexist in the same container (or even share some back-end connection) as long as the principles of variability, evolvability and modularity/composability are respected.
So a picture like this is perfectly correct (it is topologically equivalent to the one above):
So what kind of "tools" would you like to have to help you write the ugly LC code? transformation, routing, security, reliability, transaction, orchestration languages... yeah, you get the picture pretty much what a modern Service Container... sorry... ESB has to offer. Not to mention that an ESB should help you, must help you, invoke the "internal" interface of your code which can use all kinds of transport and message formats (how about a proprietary string format over a socket connection). For instance, WCF is a service container. Now if you write a bunch of "new" code to implement your services, you may not need an adapter framework, but the fact and the matter is, Microsoft, just added one. Now WCF, does not have transformation or routing out of the box, you have to use MSE, but that's fine. Service containers can be nested (if performance allows it), there is absolutely nothing wrong with that (on the figure above, it would be like having two LCp areas for instance). It is not a problem to nest capabilities, they are logically independent.
Let's talk a little bit about HTTP. HTTP gives you the "external" interface interoperability (Pe=Ce=HTTP uniform interface, all interactions have the same external interface). HTTP provides nothing out of the box to support the "internal" interface interoperability. As an external interface HTTP has got everything wrong when it comes to being useful to build connected systems: it is unidirectional, synchronous, client-server, and does not offer any mechanism to define a contract. It was designed for Browser-To-Server communications so no pun intended. The (other) REST as a style (for constructing connected systems) is even worse, HATEOS creates an extremely strong coupling because Pi must be understood by Ci, it is even worse than CORBA, since as Tim Bray pointed out there are no tools. Not to mention that they will in addition to CORBA have to agree about the ad hoc mapping to HTTP verbs and error codes. If you adopt a "resource" oriented style (with no code behind) you also don't have much room for introducing any kind of "loose coupling" code. In addition, the (other) REST style lacks basic versioning capabilities, that's a non starter in the connected systems world (again I am always speaking when a human is not in the loop to make sense of things that have changed, Roy's REST works beautifully and achieves the highest degree of loose coupling). Can you imagine a Web where the user would have to do something special to navigate the new version of a web site? Well, this is what happens in the (other) REST style? surprised? just do your homework. Because, Steve, Stefan and Stu are going to hide that from you.
In the end the bull-shitters (sorry at this point of the discussion there is no other name) that talk about HTTP as a middleware platform (with so much agility), as an ESB killer or even a SOA killer have simply no idea about what they are talking about (Tim included). You are going to have to stuff the LC code just about anywhere in your own code, without a clean factoring. Good luck for implementing and maintaining all this (not to mention reusing it). Ultimately, they don't understand how to build a connected system, they live in the remoting world making a "call" as they feel they need. Yeah, even HTTP can do that. They don't understand why they need a bidirectional interface, orchestration, assembly mechanisms, XML extensibility or even some times the services of a coordinator.
If you want to build assets that are reusable you need three things:
- Governance
- Versioning
- Loose coupling (real one, not at the external interface level)
ESBs (or service containers) will help you with loose-coupling for sure, versioning sometimes (but I don't like service virtualization strategies, I like a lot more Dave Orchard's work on XSD's forward compatibility features which are going to be boosted in Schema 1.1) and maybe a tiny bit with run-time governance, hardly with design time.
Let me just say a word on versioning, XML's extensibility seems to have somehow escaped Tim's mind, but the coolest thing XML can do for loose coupling is not a technology neutral syntax (that's just cool), the coolest thing it can do is that allows us to create forward compatible service provider versions. So when we have Ce = Pe at t=0, we can also have a new consumer showing up and we can create Pe' such that we have the new external interface of the consumer Ce' = Pe' but, Pe' is forward compatible with Pe so Ce works too, the first consumer has no idea of what just happened. This is amongst other thing what the remoting bunch can't seem to comprehend, this is why Protocol Buffers are like pouring concrete in your connected systems.
So if we go back to our set of questions:
what kind of Service Containers do you need? It depends, it is really about helping you writing the LC code. Don't select a Service Container without knowing what you are going to use it for.
Is there only one type? No of course not, They come in different sizes and shapes: big, small, cheap, expensive, open source, appliances... BTW, last time I checked, my son's school did not start offering an ESB each time we bought lunch.
Do you need a service container at all? You bet, it is very unlikely that you will be able to build reusable assets without a service container even if you start clean with Ci = Ce = Pe = Pi. Sure enough the remoting guys will tell you they don't need an ESB to remote their calls, and that's true, because all they have ever done in their lives is building system where Pi = Ci (but that's not SOA).
The key to SOA and reusability is to factor the LC code outside your business logic and never have to go back to change the business logic as new consumers come with slightly different requirements.
Please understand that... IT cannot afford to build assets which are not composable any longer... hence the remoting bunch is sending you on the wrong path. A dead wrong path.