04/26/08 :: [SOA] What's Cohesive in SOA?  [permalink]


Some people tend to think that I enjoy creating such a controversies, but at the end of the day I don't. Anybody is of course free to say and do what they want, but we seem to have reached a point of evolution that seem to imply "as long as you can say or write something, there is always some truth to it".

SOA could very well be the ideal area to say anything you want. And boy do some people enjoy doing that. Of course, you could very well turn this argument to me, but I pride myself to keep the debate open, no mater what. My only concern is helping IT architects make their day to day decisions, whether they are strategic or tactical.

The short history of SOA is that people once thought 10 years ago to use Web technologies to exchange information between servers (that was a sensitive thing to try). It started with ebXML for B2B applications, then Microsoft and IBM drove it to focus on interoperability while some people threw BPM in the mix. We then realize we could do EAI a bit better with XML and SOAP than the clunky EAI frameworks of the 90s. Now we are reinventing a component model using SOA principles with SCA, while a new community is emerging calling all these effort "crap", and asking us to go back to the basics of the Web (and with it, going back the corniest programming model concepts).

Overall nothing really took shape over the last 10 years, sure there are successful products and technologies, but at the end of the day, you look back and you say Service Orientation did not emerge the way Object Orientation did. I have not lost hope, this one is hard, and there is also a lot more baggage (solid baggage) today than we had in the 70s.

So in this context it is no wonder why so called "experts" and "analysts" can paint an "ubuesque" vision of SOA and say anything they want while FUDing an entire industry one post at a time. I won't say it enough, SOA is unfamiliar and to the people that tell you they were doing SOA 20 years, just ask them what they used for an orchestration engine, how they were "assembling" components together and if their IDL supported "bi-directional" (let me keep the dash to really emphasize the bi) interfaces.

There are many approaches to agent-to-agent "communications"  (I don't want to say "distributed computing" as I see the goal of DC to actually distribute computing resources and make them act as one). DC has been tremendously successful if you ask me, and huge progress is still made every day.

Now, you need a "uniform interface" to communicate for sure. As I said in an earlier post there are plenty to choose from:

  • Send, Receive (MEST, WSE)
  • Get, Put, Delete, Post (REST)
  • Select, Insert, Update, Delete (SQL)
  • Request/Response, Notification, Solicit/Response, One-Way (WSDL)
  • Publish/Subscribe (JMS and the like)
  • Prepare/Commit/Rollback
  • OAGIS Verbs
  • ...

You can implement agent-to-agent communication with any of them. Each have some merit, I don't really see one better than another, just trade-offs. For instance, Udi does great things with the Publish/Subscribe uniform interface (which is a bit off the beaten paths of SOA). I could even argue that it does not really matter which one you pick, at the end of the day, this collection of interfaces shows that, somehow, at some point, you will need all the other semantics. By picking one you simply agree to layer the semantics of the others on top of the one you picked. Incidentally, you will notice that some of these uniform interface may require some form of coordination, so any "so-called" SOA Reference Model that does not surface "coordinators" are incomplete and require major redesign.

The question, the very question, is do you really have a SOA once you picked one? Is SOA about taking the current programming model and arbitrarily "remote" something using one of these uniform interfaces? Anybody would venture to say Yes? I don't think so when the question is asked in these terms. So what's more to the uniform interface?

Of course, first and foremost, SOA is about creating autonomous software agent that react to a request. By definition an "autonomous" software agent of that sort exhibits a high degree of cohesion. You can't invoke anything else from a Service than what it's interface exposes. So I hope Jim, this is not what you were trying to teach us.

The problem though is this stubborn service interface, that can't be hacked in any way, and if you have to change it, you have to version it, this is yucky. In reality, let's face it, people are used to by-pass cohesive principles all the time. They can never make up their mind about what the best way to fetch or update some data elements. There is always a scenario that requires a new query to be exposed or a new update gram to be built. So over time, low level services interfaces tend to become messy as modifications result in an expanded interface. REST found the solution to this mess, "don't use a service interface", just pretend it is not there, and keep adding URIs (which are really interface point and not resources) to deal with these constant modifications. Ta..da... problem solved? not quite. There are two problems with that: a) in a connected world, there are more than one consumers, nobody owns both ends any more. What happens if you break "the other end"? b) the other problem is precisely that the programming model should not let you CRUD from the consumer side. As long as you will CRUD, you will strongly couple the "consumer" and the "provider". This is not because you chose a uniform interface to happily continue CRUDing remotely that you are building a Service Oriented Architecture. You can tell anyone this is not true, ultimately, this is what people will realize. It does not matter if you apply good cohesion design principle as you are CRUDing, you will fail to build reusable services, not because the service can't be reused at a given point in time, but because you will break existing consumers as you try to accommodate the needs of new consumers. Above the service interface, Cohesion  is Coercion in a SOA.

The goal of a Service Oriented Architecture is precisely about using new technologies a) that support forward compatibility (XML, XSD, WSDL)  and b) creating a new factoring of your business logic (SCA, BPEL) that is not CRUD-oriented to avoid this terrible coupling between the consumer and the provider. This is why people were not doing SOA 20 years ago, nobody was doing SOA 20 years ago. Of course ultimately you hit the systems of record, so some CRUD operation is going to be performed. Of course cohesive design principles are going to be applied within a service implementation. But above the service interface I would argue that nothing is cohesive. You actually want to achieve quite the opposite with loose coupling. You want to be able to assemble your service with as many consumers as possible. You may even want the consumer to tell you a few things about how to provide the service by injecting some decision rules or a sub-service provider..., again this is loose coupling. At that level none of the cohesion principles apply. I would even argue that technically, one of the major advances of SOA , (again at the technical architecture level), is that I can now build systems in ways that let me scale, secure, fail-over,... different parts of the systems differently, against all principles of cohesion. Hence, the principles of good cohesion are of no help when it comes to design the service interface.

Now, Jim, have you ever asked how cohesion can apply to an Enterprise Information Model? Information is relational, has always been, will always be. How could you apply any cohesion principles to a data model that is entirely related? There are none that you can apply. And frankly this is also a challenge in SOA. You can't easily do join across service interfaces (REST or WSDL for that matter). It really requires that you factor your data structures in two tiers: an information business entity tier and the parts that constitute a given information entity. If you have a purchase order service and a customer service, how do you pull customer information as you fetch a Purchase Order? (Say, the telephone number of the customer or the contact person, that you would always want to be the latest information available).  Now, let's say you want to use SalesForce.com as you "Customer Service" and "NetSuite" as your Purchase Order Service. You will quickly conclude that pretty much every "Resource Centric" service is "related" to any other service and cohesive boundaries are really hard to define.

If you also look at this process (click on the picture to zoom in).

(this figure is coming from Slide 4 of Ross Altman's presentation on composite applications)

There is nothing, absolutely nothing, cohesive about it. You want to be able to create value across cohesive boundaries. Services like "credit score", "blue book value", "appraisal", "payment" can be reused in many different context: home, car, boat, ... loans. Business Information like "Loan Application", "Loan", "Customer", again brought into the process assembly in a lot more opportunistic manner than a cohesive manner. This is where concept where agility, business alignment, reuse... come to play. You are not "programming" in a service oriented architecture, you are "assembling", you are "compositing", you are "orchestrating".

Cohesion offers design concepts antagonist to Service Oriented Architecture and only lives within the service implementation. I understand why, if you CRUD all day with "services", you want (absolutely) to apply cohesion principles like people have been doing it for 40 years, but don't tell me you are doing SOA.

SOA is about the emergence of an Inter-Action Oriented, Asynchronous Peer-to-Peer programming model. Sure the old programming model, the CRUD-oriented synchronous client/server one) is still going to be used "inside" a service implementation, down below, but SOA is new, it is unfamiliar, it is based on technologies that did not exist 20 years ago, all the good software design principles that were discovered back then don't generally apply. Again, I am absolutely not surprised that you or Steve think in these terms, this is precisely the problem. Once you will unlearn some of the great things of the past, and come to SOA with and open mind, you may, understand how to use WSDL, BPEL or SCA.

So yes, I am quite mad at the "cheap" and sometimes incoherent arguments that a small and cohesive community of "experts" and "analysts" are spreading to keep the "old regime" in place. I was just as mad at Assaf Arkin, Howard Smith and Frank Leyman who argued for years that BPEL/BPML were a "Business Process" thing, when most of the semantics to support business processes were missing. This handful of people set the BPM industry 5-7 years back. Can you imagine the magnitude of the loss? The same thing is happening here today, for the exact same reasons: closed minds, cheap arguments over CIRCA 2001 pictures of SOA, a small group of buddies who point at each other to gain credibility... the recipe is well known. At least, they won't be able to tell in the future that they were "unaware" of possible alternatives to their best interpretation of what a Service Oriented Architecture is.