05/03/08 :: [SOA] SOA Data Services  [permalink]

|

Eric is definitely the kind of people I like (apologies for not tracking your blog earlier). 

Yet when I review many SOA enterprise designs, I find that creating specific entity services are most often designed and implemented incorrectly. What many are building is simple CRUD service access to data that provides little business value and causes technical problems with data integrity, performance and tight-coupling with the underlying data structures – see CRUD anti-pattern.

What you want from an entity service is business orientation (e.g. an order or customer entity vs. CRUD like operations), federation (aggregating disparate but related data), high performance (such as caching federated data) and abstraction (data structure and location hidden from the services).

Actually REST (Roy's Theory of the Web) does has something to teach us in the Data Services world. When Data is spread across autonomous systems of record (e.g. services), you need to consider 3 levels of federation:

  • navigational (from customer to order)
  • attribute level (some customer attributes are in two separate systems)
  • record level (customer records are in different systems)

Of course an information system often involves all 3 levels of federation simultaneously. The role of federation is to manage seamlessly the relationships (as in ER) across disparate systems. So for instance, in the first type, a purchase order systems is obviously going to have a bit of customer information embedded in it. Each customer will have an identifier and the goal of a federation system is to make sure that when you navigate from a customer to an order, the correct identifier mapping is applied to get to its orders.

If you use a URI as an identifier (and not as a QBE with this silly URI mapping ritual), REST provides an identifier mechanism that can avoid complex foreign key mappings.

At the attribute level, REST (Roy's Theory of the Web) has also something to teach us. If we go back to the cohesion discussion and the point I was making about "Cohesive Information Models", REST can also help to solve quite elegantly the problem of providing just-in-time information. The purchase order would hold a URI to the customer's contact information (which for a business can change quite often). It is not until you physically need to consume this information (i.e. call the contact, or at least display this information) that you would actually invoke the corresponding GET operation.

I cannot stress enough the importance of Data Federation concepts in a Service Oriented Architecture, precisely because cohesive boundaries are virtually impossible to define.

The only issue I have with the REST community (not Roy) is that they probably see the same benefits as I do, and they go all the way, they say, REST works for everything else, yet what they end up doing is CRUDing because they absolutely refuse to consider how important the concept of Inter-Action is (which is missing from Roy's Theory).

I have nothing against theories, they are obviously the right thing to do, but so many times in my career have I seen theories applied outside the context from which they were developed (no less than the Web here). The first week into my PhD, my advisor asked me to review some of his calculations. I smelled something fishy, but I was not familiar with the theory, and I told him. Remember it was a time where you could not "Google" anything. It was a time where you had to order books from a librarian and wait sometimes several weeks before you could get it. Sure enough during my second PhD (I have technically 1.5 PhD, I dropped out the second), I attended a class where they reviewed the theory that my advisor had applied and he had pretty much violated all assumptions that you needed to validate before you could apply the theory). Similarly, I was working as part of a 50 M DARPA project at HRL, and again a Professor from CU demonstrated that one of the key goal of this project could not be achieved based on the theory of non-linear systems. Yet, our system had only a few discontinuities and was "linear-enough" otherwise to get the job done with linear system controls. So if you allow me, I am quite suspicious when people (PhDs or not) apply "theories" well outside their boundaries without validating (and understanding) the context in which the theories apply.

Roy's developed his theory by looking at the Web as a collection of "resources". IMHO, he has built his theory without  considering the concept of "Resource Lifecycles" (for good reasons). He made an approximation, like every scientist does. Here is why, the lifecycle of a Web Page is very basic:

You can question whether "update" is a good thing to do for a Web page, but, whatever the lifecycle is, it is basic and most importantly, it is UNIFORM for all web pages. This uniformity is the reason for the Uniform Interface, there is nothing intrinsic to the "uniform interface" (I think Roy goes a bit overboard in his thesis with this constraint, he does not explain enough the origin of the constraint, he focuses too much on the benefit).

Here is a plausible general metamodel for resources, I think REST principles can be derived from this metamodel but it has the merit to surface an arbitrary resource lifecycle and an arbitrary inter-action interface.

A uniform interface is a complete myth when the lifecycle of a resource is different than the one of a web page. This is yet another case of a few people (not Roy) applying "theories" well beyond the context for which they were defined. It is already hard enough to come up with a useful theory (ask Newton) but thinking that there are extended areas of application that serendipitously appear here and there is, in general, pure fantaisy.

Sure enough the remoting patrol, jumped on this like a bear on a beehive, they saw yet another uniform communication interface they had not explored. Moreover this communication interface was backed with extremely scalable web servers activation mechanisms. But, yet again, they don't realize that the lack of "lifecycle" and "inter-actions" that made them fail to apply Distributed Computing communication principles to information systems construction, is going to fail them again in REST. As long as we will give people some mechanisms to CRUD the resources, we will face the same wall.

I don't think I am that lonely saying all this, but sure enough, it is not as sexy as telling people: look how successful the Web is, now imagine how successful you are going to be by implementing your information systems with the technologies that made the Web successful. Woa, can't beat that logic !