State Alignment in the connected world

The notion of state and state alignment is crucial in service oriented architecture. When a service (as a peer) notifies or requires state changes to another service, we need to make sure that at the end of the interaction, the state of each service are aligned.

There is a common misconception that says that all you need is Reliable Messaging. Aside from the fact that the WS-RM specification came very late, almost as an afterthought, let's explore why it is not enough.

Let me illustrate the point with a personal experience. Recently my credit card company claimed that my account was delinquent even though I had signed up for an automatic payment in full of my credit card bill every month. I called them to figure out what had happened, and the agent told me that they had send me an email specifying that my 6 month period trial of online statements was over and I had to confirm I wanted to continue receiving online statement. It just happened that in the email they were notifying me that they would also interrupt the automatic payment (just as if the two were tightly related).

This little anecdote shows that RM is not enough. The credit card company assumed that I got the message (otherwise the message would have bounced back) and that I had both read it and understood it (not to mention they expected my memory span to be larger than 6 months -only a BPEL engine is capable of such thing).

In ebXML BPSS, we created the Business Transaction Protocol to achieve both state alignment and none repudiation. The protocol is represented in the figure below.

 

Both, request and response are followed by two signals ReceiptAckknowledgement and AcceptanceAcknowledgement.

First let's talk about the fact that they are signals and not messages. Signals have a format specified by the specification as such they should not entail any interpretation or processing error. This means that once RM tells you the signal got to the other side, it is unambiguously known that this side will be able to interpret it. If we were to use messages which format would be open to users, we would have to "acknowledge to acknowledgement" which is not a desirable thing to do.

Second, let's look at the meaning of these signals: receipts means that the receiver of the message not only got the message (as indicated by RM), but this message was valid with respect to an expected schema. The receipt is generally signed for non-repudiation. In other words the receiving party cannot claim that it did not receive a valid message once it has sent this receipt. Now, it still does not mean that the party was able to process that message. Many reason could prevent the message to be processes: some of its content violates application business rules (not defined in schema), the system might be down, ... The application or module which is in charge of processing the message to instruct the communication infrastructure to issue the acceptance acknowledgement signal to the sender. This is called a non-substantive response, because it does not indicate what the response will be. It is just here to indicate that not only the message was received (RM), that it was valid (Receipt), but that the organization of the receiver was able to act on the message. This is what is necessary to achieve state alignment.

The figure below (from the ebXML BPSS spec) shows the message exchanges in all the possible cases.

The teaching of this story is that life is virtually impossible without state alignment. Every single "Commitment" requires perfect state alignment. Businesses in the physical world have put in place everything that is required to make commitment non-repudiable and consistent to both parties.

Now, don't get me wrong, 2 parties can always sit down and agree on "their" business transaction protocol. The problem is interoperability between dozens, hundreds, thousands of parties. You really want a standards based (open if possible) protocol, you don't want to have protocol A, B, ..Z  for a number of collaborations.

The web services community has completely ignored this problem while ebXML and RosettaNet before it solved it years ago.

To summarize:

1. Reliable Messaging (RM)tells you only that a message got to its receiver safely (and in the right sequence if necessary)
2. But, tt is not because I got a message that I will be able to understand its content, it is not because I can understand it that I will act on it.
3. Therefore BPSS has 2 signals to complement any RM protocol:
   a) a receipt signal that says that the message I received matches the agreement that we have (this message was the one I was expected as defined in the collaboration, and it had the right message format if specified).
   b) an acknowledgement signal that is returned when the message was successfully processed by the receiving application, system, ... whatever (you don't want to expose this kind of detail to the other party in general)
   
   State alignment requires at least the acceptance acknowledgement. The receipt ack is rather used for non repudiation and is not part of the state alignment question per se but helps provide feedback about what might have gone wrong. If you get a negative receipt, you know you may not have sent the right thing based on the agreement you had with this party.
   
   The acceptance ack is often called a non-substantive response. It does not mean yes or no to a request, it simply means that the receiver of the request was able to process the request (it did not get lost internally).