02/11/09 :: [SOA] A Message Type Architecture for SOA - comments[permalink]


So far the article seems to have got positive reviews, that makes me happy. I think it starts addressing a vexing problem that has been described many times, and most recently by Peter Rajsky, yet no real solution has been brought up.

As I commented, I have seen this approach succeed with an Adaptive Software registry and using UML as the metamodel for the EDM. I had designed the UML profile but left the company before the project was complete so I cannot comment precisely on what was achieved other than the consultants and my former manager told me they had succeeded. I later tried without Adaptive (for budget reason) and tried to use oaW M2M UML capability, but failed due to the complexity of navigating the UML metamodel (as I needed to pick up the UML profile elements) to generate the XML Schema.

Kell-Sverre wrote a nice post on “Business Event Message Models” in the wake of the article. He adds:

I just think that we need to model also the business capabilities and interactions that utilize the message types to get a complete set of artifacts for service contracts.

One difference is that I prefer using a common information model (CIM) as the basis for modeling the message types, rather than an enterprise data model (EDM).

First, on the EDM effort, yes it is kind of true, we had bought an Insurance EDM from a vendor so we did not have to create one. Adaptive Software has also tons of adapters that can collect your system metadata, which then you can assemble into an EDM while keep traceability to the physical elements. If you don’t have either, yes you are on your own and that’s a daunting task, but I don’t see why you have to do it all to start. You can model the high level entities first and then as needed you model the basic entities.

I think this CIM/EDM debate is in for a long time J. Just to throw my two cents, if you look at Fig 9. in the article, you will see that my vision for SOA is to create services on top of the systems of record. I have argued very often that it is even OK to start with Just a Bunch of Web Services (JaBoWS), as long as you have the vision to over time evolve this service to become an enterprise service.  

For instance you want to automate a billing business process, so at some point you are going to create a “pay bill operation as part of your Bill Service” such that your process can invoke it and users do not have to open the billing system and enter the information that will result in paying the bill. It happens that you also have 3 billing systems and the process that you are automating only needs to talk to one. So what do you do? Do you spend a bunch of time in Governance to get the funding to build an “enterprise class” Bill Service? Or do you start with just what you need? I would argue that the later is more likely than the former. Is it bad? No, provided that you provision the capabilities that will make your service versionable and compatible. When another process comes around you’ll expand the footprint of the payBill operation without breaking the initial process.

This is a very likely outcome if you do things right, unfortunately none of the vendors, pundits and analysts, never ever told you that. Actually, some of the vendors wanted to sell you their ESB just on the premise that it could help with versioning…. You get the picture. What I am talking about is not hard to achieve, it simply requires a bit of discipline and understand how technologies (such as XML, XSD, WSDL) support this approach. Believe it or not I first wrote about this capability back in 1999. I had written a much more comprehensive paper (which was never published and which I have lost) on “Extensible Object Models”. At the time, I had to fight the CommerceOne guys who had come up with an “Object Oriented XML” (nice reification again –can you be more clueless about SOA than reifying SO behind OO? Yet pretty much everyone does it…). They tried hard to push it in the W3C. I don’t know by which miracle extensibility remained opened as it was the W3C working group was considering eliminating it. I guess each actor/vendor decided to ignore it, so it did not matter to them if XML was extensible or not.

Kjell-Sverre continues:

it is unrealistic that you will be able to avoid mediation completely in your service bus.

First, I like to think of an ESB as a service container, not as something in the middle. So for me Consumers and Providers sit in an ESB, it can be the same or it can be a different one (obviously you don’t want too many but 2 or 3 different ESB that offer different capabilities and scalability models can be considered). As I said in the paper, if there is a mediation that is needed, I prefer it on the consumer side. I don’t think it is a strong requirement, but again with the right versioning strategy, meditations should be minimal. Lots of mediation is going to happen between the service provider interface and the back-end systems. That’s trickier.  

Composite services that compose business capabilities across two or more domains will require mediation between the message models

I am not sure this always true. Again, either a service is enterprise class or not. Furthermore the versioning strategy that we presented supports “consumer areas” where variations that are consumer specific can be added, otherwise if they are non breaking, compatibility will take care of that without a mediation. In that case, again, you don’t need a mediation between the service provider and consumer. As a general rule, you should avoid having a mediation. Believe me, there is enough mediation between these and the back-end systems on the provider side and towards the presentation layer (or other back-ends) on the consumer side. So for me, I reiterate that the service interface is the CIM and that you get a homogeneous set of semantics because it is based on the EDM.

I guess is what I am trying to say is that you should use a Message Type Architecture and a Versioning strategy that avoids mediation (even if it only succeed 90% of the time). Too many people have decided to bite the bullet and mediate everywhere, I think this is not a viable strategy for SOA. Mediation happens between the service interface and the system of record, not between service consumer and service provider.

Now, I’d like to comment more generally about the article. Stefan and I had agreed to restart a discussion about REST but we could not pass the “caching” question. Enterprise Data does not cache very well, who can argue otherwise? How many times a day (or a month) do you need to look at the same bill? The problem I have with arguing with Stefan, Tim Bray or Steve Vinoski or Jim webber is that the only answer to everything is REST, just REST and nothing but REST. There is nothing that REST can’t do “out-of-the-box” in a superior way to anything else. This is wrong, this is the attitude that killed our industry. I think pretty much everybody gets it today, except for them maybe. REST is a great technology, it brought us the Web. However the Web is not the Enterprise, and like any technology you have to learn how to use it and where to use it. I wish simply that Stefan and I could agree on that last sentence, clearly and unequivocally. That would go a long way.

I wanted to show in this article with as much precision as possible:

a) How Resources (EDM’s entities), Events (as an occurrence of state)  and Services fit together from a modeling perspective. No service interfaces are not randomly designed (I have seen it before and yuck, what a mess)

b)    How incomplete REST vision is when it comes to actions and events (events cannot be supported without polling in REST, you can imagine how that flies). REST was designed for the Web where the actions on a page are extremely limited and you bet a page has no event on his own. A page is not an information entity that has a lifecycle. A link is not a relation, an HTTP verb and error handling is only meaningful to a Page “entity”, not to an order, invoice, or bill.

Peter asked why I had to "fight EDA", as I explained, this is not so much EDA but the reification of SO and RO behind EO. Why would you have to move to EDA because you need an event somewhere, or simply because you need some asynchronous interaction. This is what I am fighting, this system that says you have to use uniform semantics and reify everything behind it. The message I want to convey is Resources are great, Events are great and Services are great. And by the way, when you surface them properly you get business processes for free.

Solomon Duskis commented on my post on RESTful patterns.

I also don't understand what your issues are with POST ... Adding POST to a REST call explicitly states that the action performed will create a system state change.

Yes, Solomon, we agree, this is exactly how I interpret it, but I am not sure the RESTafarians will agree with you. It breaks the “resource-only” approach that they tout to everyone that wants to listen. POST is not and cannot be about just adding a resource somewhere like a payment resource to an /order/payments/ collection to show that an order has been paid. Most often, people use POST merely as an action message that will change the resource state (for instance “paid” in the order status element). If that’s the case, then REST brings absolutely nothing to the table and removes all the major advances that SOA brought us (birectionality, assembly, orchestration, compatible versioning,...). The way people use and will use REST is as a protocol not an application protocol.

The claim that I have heard from the RESTafarians over and over is that I can model some process activity by just “adding” resources here and there in some directory. If this were true, I just want people to explain to me how I answer the question "was this order was paid or not"?

If indeed people have to do this extra hop to the parent resource to change its status value to “paid”, guess what’s going to happen 99% of the time? They are going to CRUD the state of the order to the right value instead of bothering POSTing a payment somewhere else.. Actions are my only issue with REST. No action means no interface, no interface means no boundary (to manage, monitor, and all the runtime governance stuff that Stefan dismisses), no compatible versioning, no assembly, no orchestration,... No action means CRUD (for both action and content update) and RPC (for non actions) will dominate the programming model. This is my only issue with REST and has always been. An interface is not uniform, you want part of the interface to be uniform but it is an illusion to think that a 4 verb interface is "enough".

So yes, we can circle back and forth for another 50 years, we can keep pretending that EDA can do everything, or ROA can do everything or SOA can do everything (or OO or EJBs, pick whichever one you want). We have seen where this kind of conjecture drove us: each and every time, in the wall. I mean how stupid can you be when you believe that events can be implemented in REST or by saying that's you'll never need asynchronous and/or pub/sub interactions? This the problem, not the solution. We could also listen to the hand waver dudes that produced SOA-RM, SOA-RA and SOAML, we could think that the W3C SOA RA is cool too. In reality, as long as we will not express the nature of the articulation between Resources, Events and Services we will remain in this near death state where nothing works, everything we do costs tons of money (because of the reification), and where the pundits and analysts can claim anything they want and pretend they understand this space when it reality the only thing to understand is that we each have a piece of the same puzzle. The resources (entities) of an EDM can be projected in Service calls, yes events can be clearly marked as Event messages and no you don’t have to emulate request-responses into pub/sub event messages.

Where are all the enterprise architects? Where is the Open Group? The CBDI forum? Are they asleep at the wheel? We can’t expect any vendor to come up with a model like this because no middleware vendor can offer it today or tomorrow. Middleware vendors will stay at the protocol level for decades to come. Look at guys like Keith Swenson or Ismael Ghalimi, the only thing they want to sell you is their products and their corny vision of BPM, CIRCA 1995-1999 a la XPDL-BPML sauce. Guys, we are well into the 21st century, it is time to wake … up !