Book
12/23/07 :: [REST] REST and inter-actions [permalink]
Two weeks ago I was trying to summarize my debate with the REST community. Several posts later and many emails, my position has not changed.
1) I like REST, REST is brilliant, this is one of the best technology designs I have seen in a long time.
2) Even though REST is brilliant, it is incomplete. It has opened the way to make "middleware" aware of application semantics instead of remaining confined to "distributed computing" semantics. Prior to REST, all the bright minds of distributed computing could think about was "remoting": send/publish a message, remote a function call, remoting a method call... REST has showed us that there is a better way, a far better way to enable software agents to interact with each other. "Interaction" is the key here. The things that interact are not merely "sending a message", "calling a function or a method". Things "interact": the "things" and the way they "interact" represent a key understanding to successfully build "connected systems". Ultimately interaction leads to composition. The composition introduced by the Web and REST lead to presentation composition (mash ups). Other forms of composition: process and information are valuable too.
3) Because REST is incomplete, the REST community is adding here and there what they need, APP is taking the role of SOAP for instance. They don't really yet have a security, transaction and reliability story, they will one day. Can the REST community design REST-* to be better than WS-*? yes, I believe they can (not hard, just be careful who you bring on-board...). Do they underestimate the task? yes, I believe they have no idea of the task at hand. The caveat to what I just said though, is that the REST community is trying very hard (too hard?) to avoid the concept of "service" to manage "network data objects". The only reason they are trying so hard to hide services is to avoid having to define "contract" of this service. They are still in the belief that "interaction contract" means "remoting" they have missed point 2) above. One day they will also realize that they need a "contract" and on top of it an assembly mechanism.
4) Where is the REST community today? Talking with Subbu, I think I understand the disconnect. He works for Yahoo (same would be true for Amazon, Google, Microsoft Live...). For these people, who operate vast data centers, every CPU cycle counts. It speaks directly to their user base: if yahoo is sluggish people will switch to Google and vice versa. I, on the contrary, work as an IT architect. In my world, a server is not much, many large corporation critical systems can run on a couple of servers (for redundancy). Virtualization technologies let me handle seasonal peaks easily, what I care about is "weeks of developers", "risks", "downtimes" (or how my information system can deal with unavailability). CPU cycles is way, way down there for most of what we do. To sum it up, Yahoo cares about the "response time" to a "request" from a "user", I care about the "response time" to a "request" from a "business partner". I wish I could also give him or her a "sub-second" response time, but If I can offer response times in the order of a couple of hours for hot fixes, days for changes and weeks for new stuff, I am going to make him or her really happy. Believe me, caching responses is way, way below the radar (and the way people implemented REST's caching mechanism can't even reliably "represent" stock quotes or allow me to navigate pages when I am offline - how stupid are those two things?). For IT, If I can reuse a piece of code 2-5 times, this is a tremendous savings: imagine the cost of duplicating assets in IT? re-implementing, re-testing, integrating? how about maintenance? now I need to apply my changes to several code bases in different technologies?
So what is missing in REST?
1) (Re-)Establish XML as the foundation to exchange information, using XSD to manage both structure and extensibility
2) Define contracts based on extensions to a uniform interface that support bidirectional interaction patterns based on a basic composite application model. This interface MUST be bi-directional: in-out and out-in, in and out. It does not sound like much but this is essential. I know, the old remoting guys have done the best they could to lock down Web Services in uni-directional pattern: in-out and in (they have so much to loose), but that's a dead-end, and unfortunately, the RESTifarians are falling in that trap too.
3) An assembly mechanism that would let me wire components together to achieve a given interaction definition
4) A new programming language designed for "interactions"
5) Sure, and Transaction, Reliability, Security (REST has addressing capabilities) are needed too, but that's a given, REST is not going to avoid these.
The problem with the REST community is that they missed the turn and they are not on the path to build 1,2,3 & 4 because of the understandable distrust they have had for remoting technologies and that WS-* has somehow corrected. How could that have happened? you remember the little story about how Web Services were started to compete with ebXML in the B2B space? Guys, in the Ganesh's style, I have news for you. There has been a big composite system running for decades: EDI. The remoting guys had no clue but they introduced a few "features" to look like they were doing B2B: namely bi-directional interactions, XML and WS-BPEL. They tried to lock down outbound operations in WSDL with WS-I, but 2 years ago IBM introduced SCA to finally enable the binding of outbound operations, and BPEL of course if full of outbound operations ("invoke").
Now, WS-* has still lots to learn from REST. REST is packed with really good ideas, for instance things like "media type" mediation. This is a fantastic concept, but IMHO, it could be generalized further by the notion of "role" with respect to an interface: "I am invoking this interface, on this resource, with this role". Such that when I get an employee URI, I don't get to see all the employee resource content. My "role" commands the representation that I get.
Nothing that I have learned makes me think that there is anything bad about REST, I am just warning the RESTifarians that they (not REST) are heading to a wall, just as big as remoting. Just give me some time to publish this article on "loose-coupling" and hopefully, we can continue the discussion afterwards.
12/19/07 :: [REST] REST CREATES STRONG COUPLING [permalink]
Well I got what I was asking for: answers and more. Tim Bray threatened to loose his cool. With all this attention, it is time to make the final argument on REST. I was originally planning to write it in an article, but I would like to write a very short summary of this article here to keep the conversation going. The article is about Revisiting Loose Coupling in 2007.
Before I start, I would like to reiterate that unlike Stu says, I am not against REST. I am a REST + WS-* guy (not a REST | WS-*). I also want to reiterate that have no emotional or financial links to WS-*. I was originally an ebXML guy and believed that it could do a lot of good in the world. Microsoft (later helped of IBM and others) had the great idea to set out to kill ebXML by launching the only 3 standards "you'll ever need": WSDL, SOAP, UDDI. We all know the end of the story: WS-* was completed in 2007, 6 years after ebXML had pretty much delivered a similar (and somewhat more innovative) stack. If you have any doubt about Microsoft's motivation with respect to ebXML, I would recommend that you read this report from John Markoff, correspondent of the NY times in the Silicon Valley. It should echo well with what happened this year at ISO in relation to the Office Format. Note that Microsoft was not alone in wanting to kill ebXML. Ismael Gallimi and Assaf Arkin wanted to do the same with a glass of WSCI.
My issue with the RESTifarians is that they are attempting to do the same coup "All you'll ever need is REST", of course now APP creeps in, and Harry Pierson (alias DevHawk) explains how to add some durability to REST. Harry is funny: he seems to have forgotten that if we are where we are today with WS-* it is 80% the responsibility of Microsoft. I am glad you guys are disillusioned by your own product strategy -I think a lot of people are too. So now, that we have a stack that kinds of works, I would like to avoid the ebXML syndrome and have to throw it all away because some people are on a "mission".
Stu gave me his answer to the most important question: is a uniform interface enough? can we do without additional operations in the contract? Stu agrees with my argument (?). He then continues on by saying that WS-* (which includes SCA for me) is not going to do it. I think I disagree with this point and I will give my answer in the article. He however finishes this discussion by saying:
due to the uniformity constraint, RESTful services naturally have a lot more loose coupling between service implementations than if we defined our own semantic community for service interfaces that include actions unique to a particular business domain.
Stu, I may misunderstand your point but you seem to say that one thing (we need actions) and the opposite (a uniform interface gives more loose coupling, therefore don't use actions). Anyways, it is not too important, I really want to comment on that point tonight: loose coupling and REST. Hopefully, we can all agree after this discussion about a number of things.
1. Defining Loose-coupling
The goal of loose–coupling is to create autonomous components that are modular enough that they can be composed into different solutions. These components should be able to evolve (to a certain degree) without breaking the solutions that rely on them. Finally, components should allow for a certain degree of variability, i.e. they can participate in more than one type of solutions, and at least in more than one solution.
This leads to the four tenets of loose-coupling:
- Autonomy
- Modularity
- Evolvability
- Variability
2. Visualizing Loose-Coupling
(Please click on the picture to magnify)
This picture represents that loose-coupling is about making two (or more) pieces of code interact with each other by achieving these properties:
- either the consumer or the provider can operate without each other using independent technologies and can be developed at different time by different groups of people
- they can be composed together to perform collaboratively one or more units of work well after they were implemented
- they can be evolved independently of each other to a certain degree
- they can each participate more than one solution, together or separately
The picture represents a provider and a consumer. They are both peers, the consumer is the one who initiates the interaction. They have both been implemented at a different time (a mash up is a typical example). Because they have both been developed at a different time, they both assume an "internal" representation of the other side. This is their view of the interaction. Ci is the consumer internal interface and Pi the provider's one. Since they are developed independently we can never assume that Pi = Ci. So architecturally we introduce a second set of interfaces: the external ones: Ce and Pe. These interfaces are isomorphic and are defined to work together, at t=0, Ce = Pe.
Now, there is a bunch of things that happens to achieve loose-coupling (LC) within the consumer and the provider to bridge Pi with Pe and Ci with Ce. This is represented by LCc and LCp (LC = Loose coupling). Note that in this picture "loose-coupling" is achieved on the edges. I like this model better, but nothing says it could not be centralized. Loose-coupling is really achieved when LCc and LCp are achieved with minimal effort. Tools are good, but code is cool too, it just has to be as painless as possible (compared to going back to the implementation of the component and modify Ci and Pi to match with each other).
3. How do the different technologies achieve loose-coupling?
So now, that we have a model to achieve loose coupling, let's do a little bit of investigation.
- Loose coupling and CORBA: CORBA implies that Ci = Pi, so no wonder, this is probably the highest level of coupling you can imagine (Actually RMI is the strongest since you don't have the platform independence). Yeah... CORBA is bad - let me get a round of applause.
- Loose coupling and REST: the argument is more
subtle. Let's start with the case when a human is in the loop. In
that case (think mashup), Pi is actually running in the consumer, so
this is the lowest degree of loose coupling you could ever achieve.
Yeah... REST is good, let me get a second round of applause. (RESTifarians are
cheering in the background).
Let's look at the case when no human is in the loop. So what REST means is that Ce = Pe (Whatever the component is). This is super loose coupling: you can connect any component to any component, they can talk. (RESTifarians smiling). Ah... wait a minute, what happens to Ci and Pi in REST? well they are undefined. You mean that if I change the implementation, I can't really test that Pi or Ci has not been impacted? Hum.. so how does a REST implementation work then? In effect Pi and Ci exist (no matter what, you can't write the implementation without an understanding of the other side's behavior), this is what Stu agrees too, but Pi and Ci are not explicit in any way.
So, NO Stu, RESTful Web Services indeed offer a coupling worse than CORBA, much worse because at least with CORBA you have an explicit interface to re-write the implementation on the opposite side. So in REST, if Pi changes, someone has to communicate to the other side (pick your method: email, blog, telephone, SMS...) what has changed. But again there is no machine readable way to do that. Worse of the worst, you can't test the two components independently because there is nothing to test against, you have to test the two implementation together. You can't version, because there is nothing to version. Even wors-er than worse, you have to code and recode standard application semantics such as actions and events protocols on top of the uniform interface. For crying out loud, when will you put your glasses on? REST is the absolute worst technology to develop loosely-coupled distributed components. (and Don, XQuery + XML is no better). As a matter of fact, thinking that any uniform interface is going to do the job is the biggest fallacy of the decade. Eliminating Pi and Ci is the worst architectural choice you can make. It means a) your implementation is directly wired at the Pe and Ce levels and b) you constantly rewrite application semantics protocols on top of this uniform interface. REST gave you the false impression that it worked when a user was in the loop because Ci(Pi) could be constructed mechanically without effort. (RESTifarians crying, loosing their temper, cursing me, vowing to burn the heretic). Guy, this is no heresy, this is a fact and you know, facts are stubborn: our world is made up of actions, it is not "uniform". - Loose-coupling and Web Services: if all you do in your day to day activities is to use WS-* like CORBA, or if all you understood when you read the WS-* specs was <CORBA/> please go back to bullet 1). If however, you want to spend some time understanding what XML, XSD, WSDL, SCA, BEPL, WS-CDL (and ebBP), WS-TX(and WS-CAF), WS-Security, WS-Eventing... are about and can help you minimize LCc and LCp, please read my article in January.
4. Conclusion
So where does this leaves us? I hope that by now the RESTifarians will leave the REST of the world alone. They will understand that they have a lot of work to do, very hard work (not just to establish a robust communication infrastructure), to come even close to what WS-* has to offer today (not tomorrow, not in ten years). I stand by my arguments, now: shoot or shut up.
I have a simple solution for you, it is not perfect, but I can live with it, and I am sure a lot of the common folks like me who are trying to do their job day in and day out will agree. This solution is called REST+WS*. I will detail it in my paper early January (Stu I will also respond on choreography -I am not caught up on choreography, choreography is just another way to express Pe and Ce in a single artifact. It also adds some sequencing of operation compared to WSDL alone).
A word for Tim. Tim I actually don't mind if you loose your temper, if you had any idea of the magnitude of the FUD created by the kind of crap your are saying these days, you probably would think twice about saying what you are saying. I respect your contribution to XML immensely, but I am very sorry to read the kind of things you have been writing lately.
12/17/07 :: [REST] The Very Dark Side of REST [permalink]
You probably noticed that the REST community has stopped talking with me. John Heintz had the integrity to go to end of the discussion on discussing what is required to establish a shared understanding at the action level when a human is NOT in the loop. The fact and the matter is that you CANNOT DO WITHOUT A CONTRACT to establish the shared understanding.
I also appreciated the discussion I had with Teo Hui Ming on discussing whether "Search" was RESTful?. I really enjoy this kind of discussion. My understanding is that Teo has to bring APP in the picture (a spec !!!) to answer the question, and yet could not answer it completely. The fact and the matter is that a Result Set IS-NOT a resource.
These were the only two people that had the courage to go to the end of the discussion even though they saw some limitations to the REST approach. Others, have stopped all communication as soon as they understood the problems with REST.
This week-end, Amazon published a great little service to complete its AWS collection (I am wondering if soon we are going to have Service Fashion Shows where Amazon, Google, Microsoft and others will showcase their work).
That was enough to wake up the inquisition lead by the grand priests of REST who were absolutely outraged at Amazon's latest heresy. Bill de Hora, Stefan Tilkov, Assaf Arkin, Subbu ... were all ready to burn the heretic who had committed one of the worst crime in REST history. Funny, how mechanical their reaction is: never discuss the questions always point at what not to do and occasionally what to do. There is never a single inconsistency in a religion -by definition.
Subbu who was recently caricaturing my arguments by saying that "REST does not support Web Services Choreography, and hence REST is unfit for the enterprise ", discussed today the SimpleDB in the context of action based API. He learned his lesson and decided to avoid quoting me, as if I had never discussed the relationship between REST and Actions. His argument today was impeccable. He demonstrated brilliantly how REST should be used, when it can be used. I totally, absolutely agree with him that when you need to update the "content" of a resource, you MUST NOT use an action oriented interface. Unfortunately, he wanted you to come to the conclusion (insidiously) that "all action interfaces" are bad (Nice try Subbu). The point I have been trying to make all along is that the actions should only be used to change the state of a resource. Even when a human is in the loop, REST is not that stupid, it uses links (not PUTs) to update the state of a resource, but when there is no human, magically, you don't need actions anymore. For those of you who are not convinced yet, I suggest that tomorrow you try to drive your car with a (GET,PUT) interface (no links allowed) and then you tell me how you felt: : a state machine is a state machine and there is no way around it, yes, Bill, the order in which you invoke actions does matter.
Now back to SimpleDB. The RESTifarians are so religious about their belief that they cannot see the obvious. They missed a couple details. The first one is the API's version:
It has been notorious that REST is really bad at versioning (I am preparing an article on this topic that will be published early January). Of course the RESTifarian say that there is absolutely no need for versioning since there is only ONE interface (GET, PUT, POST, DELETE) and actions don't exist. Have you tried to bake in versioning in a RESTful resource access? you mean that the URI of the resource depends of the version? Ouch...
The second detail they missed is that Amazon is probably going to publish BigDB at some point and maybe they will want to develop a true CRUD, SQL based API. Have you ever tried to implement this kind of API in a RESTful way? huh? you mean you can't?
Guys, REST alone is a sinking ship -Amazon has given up on REST where it does not work- , you are misleading yourself and wasting everybody's time in the process.
12/14/07 :: [Other] An answer for Sergey [permalink]
Sergey (from the infamous 10 questions to Steve Vinosky) asks:
Sergey, I apologize for being that condescending, but when someone writes:Is it about REST vs WS-* ? No, it's about [Do-it-yourself] vs Kits
I'm wondering, am I the last one who has understood this somewhat obvious thing ?
I don't think it's going to stop people from actually doing REST across the enterprise. I believe one can do. After all, RESTful approach is Turing-complete (I don't remember what exactly does it mean :-) but in this context it means one can do any type of enterprise service with REST).
he should really question his ability to identify obvious things. Thinking that there is anything "obvious" about SOA is really naive. This is precisely the problem, people relate to SOA as they have already known it and done it for years. I know, this is how the brain reacts to the "unknown", it maps it to familiar concepts and then performs a diff. Most people either associate SOA to something they liked or hated. For instance, all the RESTifarians see it as <CORBA/>. SOA is unfamiliar, very unfamiliar, you should question everything you have learned so far when you enter the world of SOA.
I actually admit that I don't know if your statement is true or false (it is probably true), I don't mean to pick on you, the argument has been used before, this is why I am picking on it (not you). Assaf Arkin pushed back on anybody's request to add semantics to BPML on the premise that it was already Turing Complete. Same argument came about the Pi-calculus later in WS-CDL, since pi-calculus is Turing-complete we don't need anything else. I also heard that at eXcelon when one of the developer wanted to use UML activity diagrams (1.4) as a business process model. I simply have seen the dramatic consequences of this statement which basically kills all discussion while capping any evolution of the programming model. "I am complete, therefore I am done", end-of-debate. Please, let's not use this argument. Let's use our neurons instead.
Now, back to your question, No, the answer is NO. The REST vs WS-* debate is about composition. WS-* is not just for generating code (or the other way around, generating WSDLs from code as some people think it is a better approach). WCF is the prototypical example of code overtaking SOA. No wonder that these guys are waking up today and asking themselves, what have we done? we could have got the kind of interoperability we were seeking way back in 2000 with REST. At the end of the day, across the industry, Microsoft is the only company that is left without a composition model. Ouch. Worse they trying to convince everyone that SCA is all about Java and has nothing to do with interoperability (LoL). Even Sun (who has been living in the wonderful 90s up until recently) had the foresight to repurpose JBI to act as an assembly mechanism and define a good alignment between JBI and SCA.
Please check this article from Boris Lublinsky on the SOA programming models to see how "composition" is different and not a given at all in any so-called "SOA Programming Model".
At the end of the day, there is not a lot of value in code generation and I would have agreed with the RESTifarians: if this was all WS-* would bring you, we would not need the complexity of WS-*.
No, sadly enough, in 2007, even after SCA, WS-CDL, BEPL, WS-TX (WS-CAF was the better one that a handful of idiots had to kill -not surprisingly, always the same people) and ebBP, most people don't see that there is an underlying powerful composition model that can really change the way we develop, deploy and consume software.. In 2003 at the PDC in LA, I really thought that Microsoft had got it, they started to talk about "Connected Systems". For me, it was a natural evolution: understanding how to "connect systems" was the (next) problem to solve. In 2005 I spoke very briefly with Steve Shwartz and Don Box about it, but it was clear then that "connected" was just a marketing term for them. By that time, Microsoft had trashed their SOA message and the SOA section on MSDN was reduced to one page that you had to Google since it was not even linked from any other MSDN pages.
If I may, I would like to draw a parallel. When OO started to become mainstream, I know a lot of people that were developing their own "OO" runtime in C using functions with handles, because C++ was not good enough for whatever the reason was. For sure OO was unfamiliar then. I had the privilege to teach an Object Oriented Programming class for Seniors at the Faculte des Sciences de Luminy (ESIL) in Marseilles, home of the Prolog programming language. In 1993, my students had never heard of OO before (the web was nascent). I was very touched by their dedication (I am an easy professor, I don't believe in "grading", I believe in "teaching") because repeatedly they spent nights and nights doing the projects I was giving them, even though they knew they would easily get a passing grade. Those guys were already rock solid developers, but for them, there was the before OO and the after OO, they could not think without OO any longer. They were working so hard at their projects not because of me, but because their mind was captivated by this paradigm shift. Just like it captivated my mind when I first sat at the keyboard of a NeXT computer and used NeXTStep in the summer of 1991 at Penn State. I sincerely hope that Composite Software will captivate people just as much.
Today, the silence of the RESTifarians tells me that the door to composition is now wide open, THEY were the last line of defense of the "old' connected think'in", now, you can choose to enter or ignore that door and stay behind. There is simply no turning back. We have wasted enough years already.
So please, let's stop this discussion, if you want, and let's get back to work. The way forward is a programming model that offers the best of REST and WS-*. This debate is history.
12/14/07 :: [Other] REST and Search [permalink]
Assaf has some issues with my statement:
Perhaps we’re just not talking about the same resources.
This is also a question on GET with respect to “search”, can “search” be modeled in a RESTful way? I am not sure the answer is yes, but that’s secondary
Obviously we’re not talking about the same resources.
I have made this point a few times, without attempting to substantiate it. You probably noted that by now that I like precise semantics, if you want to be sloppy writing code you can, but it will come haunt you. Using semantics for what they are for usually will benefit you. Actually if even the RESTifarians are pretty uptight on some semantics (Verbs or idempotence for instance) -and that's a good thing.
So how come search could not be RESTful? I have been doing a lot of reading recently in the REST documentation. Section 3.2.2 of the RFC2616 specifies how the HTTP scheme is used to locate resources:
http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
There is a query there, but it is not specified anywhere else in the HTTP spec. The spec does however says:
since some applications have traditionally used GETs and HEADs with query URLs (those containing a "?" in the rel_path part) to perform operations with significant side effects, caches MUST NOT treat responses to such URIs as fresh unless the server provides an explicit expiration time.
So they are definitely different than your average network data object (a.k.a one of the two types of resources identified by the HTTP/1.1 specification). This is not sufficient to prove this is not a resource, since there are legitimate resources which value can't be cached (e.g. stock quotes, obviously Yahoo does not treat them as resources because I have to refresh my browser cache now and then -thought that Yahoo was a model for RESTifarians?).
Before you jump on your keyboards to reply to this post, let's continue the exploration. I know you are going to tell me that the URI is just a bunch of characters and the results return by the query is a "resource".
So let's take this argument that "the resultset is a resource". As I see it, there is a real difference between adding an element to a collection and adding an element to a result set. A collection is a resource, a result set is not. What would it mean to "DELETE" a resultset resource? What would it mean to PUT and POST? you mean I could change the content of the result of a query without acquiring the "resource representations" first? How are the semantics of returning a single resource versus a collection of resources defined? What is actually the structure of the resultset collection when there are more than one? This structure is defined nowhere, it is ad hoc.
The other problem introduced by "search" is that I can find an infinite number of URI to address to a resource (using a (true) | (false) predicate). The HTTP spec says, using more than one URI should be avoided. Since "U" does not mean unique but uniform, we are ok, but the fact and the matter is that if by mistake I create a link to a resource using a query URI /person?nickname=JJ, what happens if this value changes? well the link is broken. At the end of the day, query-URIs destroy the notion of identity which is so critical to the resource concept and to the web at large.
So for me, guys, the "resultset" IS-A "resource" argument is very tenuous. It is more an I-SAID-SO-IS-A relationship.
Sorry for splitting hairs, but semantics is what information system construction is all about.
12/12/07 :: [REST] Not even a scratch [permalink]
Assaf responded to my posts with the title "Ok, I'll bite".
Higher up the stack we find things like RESTful BPEL. RESTful BPEL is one particular way to express those higher level state transitions, like “let’s go from pending to approved”. It lets you say things like “this GET returns ‘pending’, and so the next PUT allows changing to ‘approved’ or ‘rejected’”. That sort of things, which I guess is what JJD is talking about.
Yes, this is exactly what I am talking about. I have absolutely no problem to think that at any given time the resource representation is only able to expose the subset of actions that are allowed at the current state. After this is what happens every day in billions of pages and users have no problem selecting the correct action, because they are human. This is how I interpret that REST is managing the application state via resource representation.
Now take the human out of the loop? How a software agent acting as a resource representation consumer would make sense of these actions if there was "no shared understanding" defined a priori.
So Assaf turns around and say
Resources exposed via well-groomed endpoints and entities work, I hope we’re all in agreement with that. Unfortunately, it works in a world of application silos closed off by WSDL service definitions.
Again, can someone in the REST camp tell me how "a resource representation consumer would make sense of these actions if there was "no shared understanding" defined a priori." I have had a discussion with a hard core RESTifarian for several weeks now, and he has postponed answering that very question. We are down to it now, this is the last point we need to clear up, everything else is clear. Again, with a human in the loop, it all works magically and this is what gave us the web as we know it.
So I am becoming a bit impatient to get the answer (of course I know the answer) and I would like to hear it from a RESTifarian. Somehow, Assaf, gives out the answer inadvertently before returning to the traditional REST blablabla:
RESTful resources are in charge of their location, operations you can do against them, and their representation. Operations go beyond four verbs, that’s the very essence of responses returning even more URLs.
He admits that the resource interface is not uniform and he tells us how "additional" operations are expressed as URL, cool stuff, but when no human is in the loop, you need a shared understanding between the resource consumer and the resource (provider). You pick your favorite method to express this shared understanding:
- an email exchanged by developers
- a blog post
- a WSDL or WADL
So go ahead, guys, make my day, answer the question...so we can all stop wasting our time.
12/11/07 :: [BPM] IBM Research is working on Resource Lifecycles [permalink]
Marlon Dumas emailed me the link to the home page of Ksenia Ryndina, Ph.D. Candidate, who is working with Prof. Gall at IBM Research Lab in Zurich. This is where the Atomic Force Microscope was invented in the 80s. This is a very small facility compared to Yorktown Heights, but the quality of its research is inversely proportional.
I exchanged some email with Ksenia who confirmed that:
The results that we have published so far and the prototype we've built as an extension to WBM (WebSphere Business Modeler) address the exact problem you pointed out: "...almost no-one would be capable of designing (as in graphically designing using BPMN) a business process that would comply with the lifecycles of all the resources involved. Assuming you created such a model, let's say that you now create a process variant. How would you be guaranteed that the resource lifecycle was not impacted?"
I believe that IBM has build an architecture that is 100% compatible with the blueprint that I proposed, however, Ksenia's research confirms that they are still a tiny bit away from making it fully operational.
Ksenia, is the first person I talked to in 10 years that thought about a similar BPMS architecture.
12/09/07 :: [REST] Uniform Interfaces and Application Semantics [permalink]
I have had quite a discussion with some members of REST community (a.k.a RESTifarians). I appreciate the time and the passion some have invested in this discussion. Some have also stopped talking.
If you are intrigued by REST I would recommend that you read:
- the Atom publishing protocol this is by far the most complete RESTful application I have seen documented
- RFC 2616 The Hypertext Transfer Protocol HTTP/1.1 (specially the section on verb semantics, people are still pretty much confused about these semantics) and every one writing a RESTful application has to go to the ritual of VERB-Action binding.
During this discussion I think I have established that:
a) the way the RESTifarians think about resources is incomplete. They simply ignore resource lifecycles and actions. This is, IMHO, the worst aspect of the RESTifarian community, their fundamentalism will set us back 30 years. They are so used to have a human in the loop that deals with actions as embedded links in the resource representations that they don't see it anymore. In service-to-service interactions, the RESTful code that invokes and implements these actions does not rely on a contract. So once you built your service consumer and provider if something needs to change at the contract level you have to figure out a way to communicate that to the other party. Again, works great when a user is in the loop, but this is catastrophic for a service consumer and provider. At the moment Google is updating gmail frantically (maybe it won't be beta anylonger?), everyday I seem to do something differently than the day before. It is annoying but I can still use gmail because I can make sense of it. Now transpose this scenario when no human is in the loop. A uniform interface does not help at all when the "action" interface has changed in any way.
b) RESTifarians seem to be aware that there is indeed an implicit contract but they don't pay attention to it. People like Bill de Hora even think that the order in which operations can be invoked is unimportant. They claim that a contract buys you only generated code and they do not believe there is any value there. (I will write a paper about "Loose coupling revisited, because obviously these guys don't seem to understand what loose coupling means when a human is not in the loop).
c) RESTifarians use HTTP as their middleware platform. The REST uniform interface is a middleware interface. There is no such a thing as an application level uniform interface: only a fraction of the application level interface can and MUST be uniform (I'll be writing more on this topic). This middleware interface is smart(er) and knows about the nature of things that are carried through it, but it is only and for ever a (better) middleware interface. This is why they always go through the ritual of writing down in a document the mapping between their CRUD and Action interfaces and the REST uniform interface.
As I mentioned, there are multiple uniform interfaces possible (ignoring whether it is connectionless or connection-based):
- GET, PUT, POST, DELETE (REST) | Resource aware
- Send, Receive (MEST) | Resource aware
- Send, Receive, Request/Response, Solicit-Response (WSDL) | Message aware
- Write, Read (Stream based) | Not aware of anything
- Create, Read, Update, Delete (RDBMSs) | Resource aware (Tables)
- Publish, Subscribe (MOM) | Topic aware
- Microsoft Robotics team has yet created another uniform interface by mashing them up | I think that it is at least resource and event aware
Whether you consider it smart or dumb middleware, any of these interfaces require lots of code on each side to implement complete application semantics especially the one that deal with resource actions.
I would argue that after reading the HTTP/1.1 Verb semantics, I feel that RESTifarians make a different interpretation of GET, PUT and POST for each resource during the Verb-Action binding ritual. They also claim that a URI is a URI and it has no meaning, it is just a unique character string. In reality they use consistently the URIs to map their action interface, as well as the "search" interface.
I would like people to note that HTTP as a middleware platform is vastly incomplete because it is build on the assumption that a human is on one side able to compensate for any communication error (or application error). For instance, Harry Pierson gave us his solution for implementing Durable Messaging (Harry, could you explain how you do transactions in REST? durability is not enough). Mark Little provides some more perspectives to the RESTifarians in the context of compensations (and transactions). I must admit that Harry is hilarious. Harry says that (and this is not innocent coming directly from "the voice" of the architects at Microsoft):
like many I've become disillusioned with the WS-* stack over time and see REST as a viable alternative to all that spec-driven complexity
The first thing he does is to point us to a "spec" to solve this one problem. Who do you think you are fooling? Turns out that this spec is written by Dale Moberg and Reed Drummond who happen to really know the difference between writing a spec and writing a blog post (which is an extremely rare capability in the REST community). Many might think that we travel the world, wine and dine all day and occasionally we spit out a cryptic document (making sure that the least amount of people can understand it). Well, there is actually some truth to it, these are often the same people (a.k.a Roman Generals) that only think of technical committees as being a (political) battleground that is going to serve their career. but at the end of the day you also have hundreds of people that are passionate about sharing and understanding and ultimately advancing the state-of-the-art of software engineering, if not computer science. Harry's post is an insult to these people that have often made important personal sacrifices to contribute to this work that benefits all.
d) RESTifarians often use an Amazon as their showcase. I would recommend that you look at this presentation from Amazon's CTO. In the middle of the presentation he goes through the history of the platform and explains how they achieved the kind of scalability they needed (that most people don't need) via RESTful services. I would challenge however the RESTifarians to convince Amazon to explain us how they use RESTful services in service-to-service interactions.
I think at this point for me the debate is closed, I don't think the hard-core RESTifarians can be convinced, they do not believe in contracts and assemblies as core technologies that enable a higher degree of loose coupling (and verification). I explained in this article how resources, actions and assemblies form the foundation of a loosely coupled process centric programming model. This is also the nature of wsper. So if you have two different set of beliefs and two different problem contexts, I don't see why we would agree on a common solution. Actually, it is more "they agree" on anything than "I agree", because as far as I am concerned I am all for making resources explicit (all the way) and I don't mind smart(er) middleware as long as it lets me implement application semantics easily. REST alone as defined by the RESTifarians does not fit in this category.
I am however extremely distressed at what I consider an irresponsible attitude from a small group of people that want to impose their diktat on the industry with more than questionable arguments. I appreciate the fact they do not want us to spend money on expensive middleware. I support that aspect of their initiative, I often advise people to think clearly and precisely about the quality of the communication infrastructure they need. REST works because most often there is a human on one side that can understand that the other side is not in the state it wishes it to be. A human can deal with unreliability far better than a software agent. State alignment between a service consumer and a service provider is one of the hardest thing to achieve, and HTTP won't give you that "out-of-the-box".
As a middleware platform, REST is missing orchestration, assembly and choreography concepts (which combined offer an unprecedented composition capability, no middleware platform ever achieved this). RESTifarians simply don't get it, they never spent the time to reflect on these concepts. I strongly believe that they have missed what happened in the last 3 years in the SOA / WS-* world. I can appreciate that around and before 2004 some lost faith in what WS-* was going to deliver, but I strongly recommend you go back an look (most of you still don't understand that WSDL offer a bi-directional interface with both inbound and outbound operations)
Worse, RESTifarians (because REST mixes application and middleware semantics) want to teach us how to construct an information system in a way that will negate any progress made over the last 30 years.
For me REST + WS-* is the way to go (even if it means some things have to change in WS-*), both WS-* and REST can learn from each other, for the greater good of everybody, without killing each other. REST alone is a dead-end. As we speak, people are already building REST-*, and they are totally unappreciative at how complex creating a specification can be (they have never been in a standard committee). At the moment they simply exchange posts, tips and tricks, and voila. It will take another 10 years to transform these posts into specifications from the time Microsoft, IBM, SUN, BEA and Oracle will get involved. In that case, see you in 2017...
In conclusion, I would like to express that fundamentalism has always and will always be on the path to progress because it demands to apply consistently the same solution(s) regardless of the problem at hand. I often speak of the "fundamental problem" or the "fundamental question". You will never catch me saying "the fundamental solution". This is an oxymoron, for any given problem there are only good and bad solutions.
Since the advent of the web (and thank to REST) we have understood that monolithic information systems can be thing of the past. (SaaS, IMHO is perpetuating monolithic architectures.) The value of composite information systems is compelling not just for scalability reasons, but also for pure economical sense. For instance a composition information system should be able to reuse assets that can't be replicated on-premise (Google Maps). Because composite information systems are desirable, middleware and application semantics need to come closer and offer the best layering possible (distributed objects being the worst layering possible). The RESTifarians are sending a strong signal to the WS-* community, ignoring it would be being just as fundamentalist as they are. A door is opening, driven by strong value. I am actually predicting in 2-3 years time people will seriously question the value of "desktop-based operating systems" because their "personal resources" (and their lifecycles) will be exclusively handled by composite applications. Which side of the door to you want to stay? The RESTifarians (and some vendors) are trying to convince us to sweep a decade of painful progress under the REST rug. Yet, we have never been closer to opening that door. I am sorry to say that your irresponsibility is only driven by ignorance and belief.
Now, thinking that anyone has "broken the code" of composite applications in 2007 is premature, there is however no turning back from this point.
P.S.: I have never really talked about how hypermedia links related to "resource associations" but I spent quite a bit of time to think in this direction as well. It will be part of an article I am writing.
11/20/07 :: [SOA] REST, Processes and Resources, [permalink]
Assaf pointed out an interesting article from Micheal zur Muehlen et al that explore the question of REST vs SOAP in terms of process integration.
IMHO, the article is focusing the discussion on the pros and cons at the protocol level.
The question I am raising is quite different, it has nothing to do with a "protocol", in the end you can decide that REST "as a protocol" is better than SOAP (I don't care about this answer, and really the answer is "it depends"). Of course the RESTifarians are outraged that this question could even be asked -and I understand why, because this has nothing to do with REST.
The question I have is on the pure application semantic level. The question is Does a resource have a lifecycle? if yes, there is a follow up question what is the understanding that needs to be shared between a resource and a resource representation consumer to transition from one state to another? So to be clear, the question is not on GET but on PUT. I also argue that the question's answer is quite different depending on whether a user or a system is the resource consumer. (This is also a question on GET with respect to "search", can "search" be modeled in a RESTful way? I am not sure the answer is yes, but that's secondary).
So if indeed, resources have lifecycles, and if indeed you need a shared understanding of "actions" expressing your intent to transition the resource from one state to another, I would argue that the debate REST vs WS-* is really at the protocol level. They are just two ways of encoding the same semantics. When a user is in the loop, REST wins hands down, if we are talking server-to-server, I would argue that WS-* wins hands down except where Amazon-like scalability is needed.
Now, since the article from Michael and his colleagues talks about "Process integration" I would like to clarify my position on the relationship between "Process" and "Resource lifecycle". This is not about process integration but rather how do you factor resource lifecycle within a business process. Here is a Job Application Process modeled using BPMN (click on it for a larger view).
As you can see, the states of the Job Application resource are not visible in BPMN (nothing wrong with that, this is not necessarily a concern of the process designer, this is rather a constraint for the process implementation). More importantly, the activities performed by the participants are orthogonal to the actions invoked as part of these activities. The activity boundaries can span multiple actions or match exactly the actions that will result in state transitions.
The job application service is assembled with other services (such as human tasks) to perform the process.
In effect a process is merely an assembly of services (click on the picture to zoom in), i.e. wiring service operations (both inbound and outbound) with each other:
This why SCA is so critical to BPM. A process container is an SCA container. Steve, the very reason you say:
I believe that it’s important to avoid specialized interfaces whenever possible and prefer a uniform interface instead. Interface specialization, even of the minimal variety you mention, inhibits reuse.
is because you don't understand the concept of an assembly and what it does for reuse in a server-to-server scenario. It is not by moving to a minimalist interface that you will promote reuse. You cannot ignore the semantics of an action. The solution that you are recommending to improve reuse is the worst I have heard in the context of server-to-server communications. You are pushing the reuse problem at the developer level: "figure it out", read the doc, script the reuse.
In this picture, BPEL is the implementation language of composite services such as the Job Application Service.
As you can see, we traveled quite a bit of ground in a few lines. Resources and resource lifecycles are at the foundation of composite application models. At the end of the day I can't care less about which protocol the REST community, Microsoft or IBM wants me to use, all I care about is having the correct application semantics to build composite information systems. That's all I need, that's all I want. If you want a more detailed discussion, I have written a minibook on the subject available to download on infoQ.
Sadly, and I repeat, the thought leaders that define programming models (Assaf was one of them in the BPM space) have ignored systematically this factoring. Resources and Resource lifecycle remains a concept that you have to code over and over no matter which programming model you turn too (including Erlang). After almost 10 years of "thinking" on Service Orientation this is the best thinking you get:
Don Box: Personally, my dream stack would be ubiquitous WS-Security/WS-Trust over HTTP GET and POST and tossing out WSDL in favor of doing direct XML programming against payloads from VB9 (or XQuery), but hey, I have unusual tastes.
Assaf Arkin: The orchestration landscape is a mine field of different opinions. For some, it's all about top-down static endpoints using infrastructure to exchange control messages in the background. For others, well at least me, it's about message passing between processes, and REST fits nicely in that.
Bill de Hora (speaking for the RESTifarians): So, in a business process where GET and PUT (and friends) apply to *all* business entities and are not just per process defined methods, why can't I GET the state and have a well-understood formal document returned citing the state of that entity? Or for that matter PUT the updated state to that entity? What's the actual limitation induced by applying REST?
Steve Vinoski: I know from significant first-hand experience what both sides of the coin look like, and there’s no question that REST-oriented systems are easier and less expensive to develop, and far less costly to extend and manage. Like Dare said, anyone who thinks otherwise is either so emotionally or monetarily attached to WS-* that they can’t be objective, or they don’t actually write any code or build or maintain any actual systems. It’s no contest, really."
At this point, I start wondering who is not writing any code. This is why I am a bit upset at the REST community, at Stefan and Bill, and others. Only you, have the concept of a resource within your programming model. This is the opportunity to revolutionize the way we construct information system, this could be a benefit almost as big as the web itself. Yet this community wastes all its time, and other's time, trying to funnel all application semantics through two verbs. That's all they care about. In the process, they have lost sight completely of what a resource is.
Again I am not selling technology, but I want to point out that the application semantics offered by the SOA technologies at large make sense, they fit with one another, and it is not because Don, Bill, Assaf, Steve and others can't figure it out that we have to throw it all away.
This whole stack is available today in the Java world (open source or commercial). The only piece missing in the .Net world is an "interoperable" assembly model. SDO is available in .Net. XCalia has built it. Microsoft, with the help of Dave Chappell, is trying to convince you that you don't need SCA. They argue that this is a "portability" problem. Note that I am not arguing that Microsoft should adopt the programming model of SCA, that's a very different question. They have innovated and invested massively in WCF and I don't see a need to change that. But hopefully I have convinced a few more folks in Redmond (and Armonk) that they need to offer an interoperable assembly mechanism. The value is in what you assemble on-premise or off-premise, not in the assembly mechanism itself. In some ways, I don't care, because someone like Xcalia will build it based on SCA without Microsoft's control and value add. This is just a matter of time.
11/20/07 :: [SOA] REST, Resources, Lifecycles, States and Actions [permalink]
Bill, thanks for replying. I think we are at the heart of the discussion. Hopefully the pictures are not just pretty, they help illustrate and reason about this discussion.
Bill: I'm not sure what my post about RESTFaces has to do with anything [you are] talking about
JJ: To clarify, you said in your earlier post about RESTFaces: "The giveaway word is "action"; unfortunately that's not REST style, that's RPC". I am not sure my post had anything to do with business processes. I really talked about a resource, its lifecycle and the operations that can be invoked to change its state (not its content). Of course, these types of operations are often invoked as part of business processes, but I really would like to focus the discussion on resource lifecycle (states and transitions). I don't want to use concepts other than the one that REST claims you need.
Bill: It seems to me that the "business interface" mentioned is the exchange of well understood document/schema along with well understood operations. In other words you have to define both.
JJ: No, they are orthogonal. The tragedy of modern programming models (including and above all OOP, RDBMS or even the latest LinQ API) is that they do not distinguish between content and state. I actually never saw a developer clearly articulating the state machine of a business entity or an object, let alone change its state in an orderly manner. The business interface contains the operations that will trigger a state transition. Some content may be provided to decide if the transition is valid, but the intent is to trigger a state transition, not to update content, which can be achieved with a PUT (a schema would simply avoid mundane data type errors). It would be a good pattern to enforce the separation between content update and state transition requests.
Bill: why can't I GET the state and have a well-understood formal document returned citing the state of that entity? Or for that matter PUT the updated state to that entity?
JJ: You can PUT the updated state, this works perfectly fine. My argument is that in doing so, you cannot ignore the semantics of the business interface -unless I am missing something. How do you know that changing a value in the "representation" of the resource, will actually trigger a transition from one state to another? Don't you need to share this understanding between the consumer of the resource representation and the resource? How could you expect consistent results otherwise?
Back to your first point, could you please be more precise about what you mean by "a well-understood formal document returned citing the state of that entity"? I am not sure REST has any semantics which can even come close to what I can imagine reading this sentence. Are you talking about a schema? a business interface?
In general you don't want to expose the state machine, this is private information. It may even be expressed in a formalism that is incompatible between the representation consumer and the resource itself. Incidentally, the state machine may not even be expressed at all. It might just be "in-the-implementation" as it is so often today. You are generally more inclined to specify the actions that cause state transitions, this is a much better separation of concern, a lot easier to agree on, both on the semantics and the expression of these actions. You may call them RPC, but they are not (I am not calling a procedure).
Bill: Certainly there are document schema for interchange in BPM space, but they seem to be stuck on non-uniform methods, aka operations, as were the WS-* standards before them.
JJ: I think you may be mistaken. Are you familiar with the work of the Open Applications Group? They actually recommend using a very small number of verbs (9 verbs) and an infinite amount of nouns. Now, it does not mean that I necessary buy this limitation... because they are not making states explicit either. OAGi Verbs include "Process" or "Synch" (which are content based, the OAGi architecture predates REST and is some form of Business-REST). At the end of the day, I am not sure I can construct a business interface with these verbs (I have a tough time any ways), I will have to agree out-of-band on combinations of (Verb,PropertyValues) as actions, I argue that this is worse than having an explicit business interface.
Bill: One downside of non-uniform (or per process) operations is that you have to worry about the order the operations are executed in, and ultimately that reduces to a programming/scripting exercise, but happening across system boundaries.
JJ: That's what I am talking about, I am not sure you can ignore "them" and the order they are executed in otherwise you will sometimes require transitions which are forbidden and generate a fault. What's the point? Do you want to create systems that "try-until-it-works"? Note that you don't seem to be alone in that camp. Most of the thought leaders of our times simply don't get it. For instance for Don Box:
Personally, my dream stack would be ubiquitous WS-Security/WS-Trust over HTTP GET and POST and tossing out WSDL in favor of doing direct XML programming against payloads from VB9 (or XQuery), but hey, I have unusual tastes.
The scary thought is that his team is now designing the next generation programming language of .Net. I know exactly how it will look like.
Bill: Perhaps all the SOA/BPM community are saying is that REST doesn't have a formal interlingua for business processes
JJ: No, this is irrelevant, this is more profound: it is at the root of the programming model. This is about making states explicit. The genius behind REST is that it works perfectly for human-system interactions. The system can expose actions as part of the Representational State (i.e. application state as opposed to resource content) and humans can make sense of them without effort or any kind of a priori information exchange (you do need user's manuals from time to time, but for the most part they have been eliminated). So, what I am arguing is that
a) the REST camp is thoroughly denying that actions are a necessary semantic as part of the programming model
but they will always be there. they will never go away, they are part of the nature of resources (not processes)
b) REST makes things a lot more complex when two servers interact and are expected to transition a resource to a new state as part of this interaction.
you are on your own to express and code the business interface. In server-to-server interactions REST becomes a "protocol" over which the business interface is layered (painfully).
Simply put, REST benefits are not transposable to server-to-server interactions without an implicit understanding of the "business interface". A "schema" really optimizes the utilization of the "PUT" pattern for updating content.
Bill: but why would it any more than an NFA state machine formalism would? That would be bad layering
JJ: Of course, it is the other way around, you want to associate a (DFA) State Machine to a resource, not a resource formalism to a state machine formalism. A lifecycle is an intrinsic part of a resource. Any resource, especially business entities. These state machines are very intuitive, very easy to design and would be easy to leverage if only the handful of computer scientists that define programming language semantics would understand that this is a terrible mistake to think that these semantics can be derived from a smaller set of semantics because of the amount of inefficiencies that are generated by having to recode these concepts over and over.
Bill can you imagine the amount of useless work that is/would be performed in the world simply because these semantics were not important to you/people like you? It would be like talking with a vocabulary of 4 words. Now, REST does not stop here, it even goes further, it says, you don't even need the semantics of a business interface because you have a PUT. I am sure we could invent a language based on a vocabulary of two words. That's not the problem, the question is, is it what we need?
Bill the problem is not to find the simplest set of semantics - this is an interesting exercise - but by forcing everyone to adopt these semantics, you would be killing IT. If I may I would like to suggest another interesting exercise. How about finding the smallest amount of application semantics that would make it easiest to develop your typical information system?
Bill please note that I am not asking you to endorse WS-*, I am only talking at the semantic level, you can still think that WS-* is XXX. WS-* is all I have ... for now, so I take it. In the end you will realize that I am even a stronger RESTifarian than you or Stefan are since I am advocating to put the resource (and resource representations) at the center of the programming model, but in doing so, I make no assumption about the programming model itself, I try to find the best programming model on this foundation.
11/19/07 :: [SOA] Bill de Hora complains about the RESTlessNess of RESTFaces [font>permalink]
In my last post on REST I was saying that the real question was :
... the real question is how do you model an action to transition a resource from one state to another?
The REST community did not really pick on that one. I am not surprised. Let me explain why I am not surprised: the irony is that Representational State Transfer cannot efficiently deal with the state changes (content and lifecycle) of a resource. Let me prove my point.
Let's take a Job Application resource. Its lifecycle should look like something like this:
Now, Bill de Hora complains that RESTFaces has nothing really RESTful about it and marvels at RESTLets. This framework implements the concept of a router that associates URL syntaxes with class definitions:
// Create a router
Router router = new Router(getContext());
// Attach the resources to the router
router.attach("/users/{user}", UserResource.class);
router.attach("/users/{user}/orders", OrdersResource.class);
router.attach("/users/{user}/orders/{order}",
OrderResource.class);
// Return the root router
return router;
In this framework, not surprisingly, a resource class has two methods:
- Representation getRepresentation(Variant mimeType)
- put(Representation r)
The fundamental question is not so much about GET, but PUT. Ironically, the web architecture document labels put as an unsafe interaction (I am pretty sure this word was chosen on purpose, see below :-). While requesting a PUT how does the client knows:
a) the types and constraints that will create a well formed resource representation that can be "put-back".
b) (now, the interesting question) the correct states and transitions.
What's the most efficient way to communicate these two pieces of information between a client developer and a resource developer? Let me think a nanosecond before I give the answer: a schema and a business interface. In REST you have neither, absolutely, positively, idempotently neither. Yeah, you get a nice PDF or Text file describing what you need to do. If there are "multiple versions" of the content and lifecycle, well you just hope you got the URL right.
In reality, the lifecycle mandates a business interface (either implied or explicit). A business interface that needs to be both known by the "client" and the "resource". The operations of a Job Application "service" are associated to each and every one of the transitions within the lifecycle.
Mark, I don't think you can express the business interface in a RESTful way. REST says PUT is enough (I need an out-of-band contract to express it). The reality is that sure, Google, Yahoo, Amazon are ready to sacrifice the business interface to gain in scalability. There is nothing wrong with that, they have few interfaces and can easily ask their "clients" to figure it out. Their interfaces are fairly static. In the enterprise you generally don't care too much about scalability, you care a lot more about your content updates and state transitions. This is why you have information systems in the first place. You can't sacrifice anything in content update and state transitions. Since this is really important to you (you will generate a fault if something is wrong) you want to push as much knowledge to the consumer such that faults are the exception, not the norm.
The class of problems addressed by REST is far, far away from the day to day needs of an enterprise. If you have requirements that are well addressed by REST, this should be your first choice by far. However, if you need to build your next solution and need to deal with change efficiently, well REST is not the answer, by far, by very very far. So I am not surprised RESTFaces is not so RESTful afterall.
11/16/07 :: [SOA] 3.0 [permalink]
The REST debate is behind me, I did not see any argument that would make me think that the REST community understands anything past the presentation tier and I certainly found Steve's last post pretty sad and depressing.
Eric Newcomer started a new discussion topic IT 2.0. If I agree in general with Eric, I would like to point out that the SOA stack was really completed in the spring of 2007 with WSDL 2.0, WS-Policy 1.5, WS-TX 1.0 WS-ReliableMessaging 1.0, SCA 1.0, SDO 2.0, WS-BPEL 2.0, BPEL4PEOPLE 1.0 and WS-HumanTask 1.0.
One should be careful in talking about SOA adoption because, IMHO, SOA, as we are going to know it, is less than 6 months old.
People love to talk about 2.0 things. I'd like to talk a bit about what's coming in 3.0.
| Web | Consumers | IT |
| 1.0 | Basic content and basic apps | Basic presence |
| 2.0 | Rich content, search and content-based collaboration | Rich interactions, self-service and outsourcing |
| 3.0 | Get the web work for you, search for collaborators | Right-sourcing (people, process, services) |
What do I mean by "Get the web work for you"? The next revolution in social computing is to bring individual together to perform work (Mark found a perfect example, and our very own Brier Dudley is participating in a web 3.0 experiment too). Amazon has already innovated long ago, with the Mechanical Turk (I hate this picture, guys can't you come up with a better name for the web 3.0?). If you want to see something more concrete, you can take a look at SmartSheet.com. This collaborative tool is the embryo of a process engine (as a shared spreadsheet) to let people perform task collaboratively based on a process definition (template).
I have spoken many times about right sourcing. This is the killer app for SOA (REST + WS-*), this is composition at the presentation, process and information layers, this is resource optimization and agility at its best (virtualization is just the tip of the Iceberg and yes, Amazon has a small head start here).
There is another factor to take into account. COBOL developers start retiring and the picture is only going to get ugly. Even TATA and Wipro can't train/provide COBOL developers.
If you combine the benefits of right-sourcing with the retirement issue, our industry is facing a very disruptive environment which will entail a major evolution of IT as we know it. The technology is here, it is not a coincidence that so many standards came out this year... So I am not sure, Eric, that what is coming is "incremental". The companies that are launching vast SaaS initiatives as a response to their aging work force may well miss this revolution. The next five years promise to be very exciting for our industry.
I would expect that things will heat up in about 2 years as IBM, SAP and possibly Microsoft's Composite Application Platform come to maturity. I don't think the window is bigger than two years. Frankly, this is also why I am a bit saddened by the Open Source community mainly siding behind REST. REST will not get you there, I hope you guys know what you are doing, because this could be a historical mistake. I don't see any serious open source Composite Application Platform. I don't think Interface21 gets it, though they will be the one I can see that could stay in the race.
11/16/07 :: [SOA] What a strange debate [permalink]
What a strange debate it makes when a community says "All you need is XXX" and another says "You at least need XXX + YYY". The first community says, since you are not on our side, you are against XXX. Nothing could be more wrong. Note that this debate is even more surreal for me since I am a WSPER guy and WSPER's goal is to let you choose whatever you want at the infrastructure layer. I don't care. I only care that I can implement WSPER.
I respect Stefan enough to know that he is 150% sincere on what he writes, though I cannot understand that he does not spend a bit of time looking at what he can't do with REST or could do better with ws-*. I think he understands that remark:
I think Erik is right in that we should maybe focus more on what to do instead of what not to do.
I read Jim Webber's interview, were he talks about MEST, REST and Guerilla SOA:
Operations are an abstraction which I do not believe exists in a service oriented architecture
I also attended Microsoft's Strategic Architecture Forum this week (which explained why I could not answer earlier). I met with the Microsoft's Robotics team which claim to have invented a new REST-based middleware but they had to use 9 verbs and add the concept of "event" to REST. They claim extreme scalability...
So what does REST/MEST/MSR have in common? They look at a very small piece of the problem and they each claim we can cover everyone's use case. Note I did not put the WS-* in it because nobody claims you can do everything with it. Personally, I am in the "apply the most appropriate solution to the problem" camp. I am against all form of fundamentalism. I think of a fundamentalist someone who applies consistently the same solution to all problems. I am a WSPER(WS-*/ebXML/REST/MEST/MSR/...) guy.
I just would like to go back to the debate for a second and express some of the arguments I told the MSR team. Mark does a great job at focusing the debate with extreme precision:
Workflow systems try to use the machine to assist in the coordination of work between actors, that coordination often being tedious and error prone and therefore ideal for automation [don't write this code by hand]. ... Does REST do anything to help us build such systems?
... the real question is how do you model an action to transition a resource from one state to another?
You take for instance WS-CAF, one of the best spec I have ever seen (now defunct and replaced by the sub-standard WS-TX, SOA lost a lot in that swap). WS-CAF is about a generic "Coordination" infrastructure. Do I need it to build a Service Oriented Architecture? Could I built it in a REST compliant way? I believe so, since a coordination context is one of the most well-formed resource you can find. But, would there be reasons for offering an alternative implementation? yes, WS-CAF supported also a "pass-by-value" model for the coordination context. Is that REST compliant? I don't think so. But that's not the point, the point is that the REST community is looking at a very small set of use cases which involve massive scalability and presentation federation (a.k.a. mashups). When the use cases change, they have to invent new concepts like the MSR team. With WSPER for instance I set the context of "Your typical IT information system like CRM, ERP...", I don't claim WSPER is applicable to Google, Yahoo and Robots. If you can apply it great, otherwise, find another solution, always find the solution adapted to your problem (today and the foreseeable future).
The commonality between REST, MEST and MSR is that they seem to have chosen the distributed computing paradigm that fit their problem space (elegantly and efficiently), but they are all wrong in insisting that this is the only paradigm you'll ever need. The mere existence of three groups claiming the same thing should make it clear that neither is right. Frankly I was equally disappointed by Jim Webber's talk but I don't have time, energy and passion to start a debate on "sending/receiving a message is all you need". (Jim, One Ways and Notifications are "operations" in the WSDL sense).
The fundamental question that I asked the MSR team was "where is the business interface", in other words "how do you model an action to transition a resource from one state to another". Incidentally, this is all you do in information systems. This is your only goal. When you have an information system, all you want to do is to change some state in it ( add, change, cancel an order), and you must have a clear shared understanding of the state you want to change. But is it practical to change the state directly at the resource representation level or is it better to have access to a series of actions to change that state? I claim that it does not matter, the result is the same. In the UI, there is always a "button" that you push to cancel an order. Again, I can see how REST simplifies presentation federation and requires no knowledge at all from the user/client perspective to invoke this action but this is only an optimization of two identical concepts. Granted an elegant, important, must-use optimization (ok, no schema means validation hell, but let's pass on that). I get the caching requirement that you can't apply in the WS-* world, but that's also an optimization. I could however understand that REST would claim superiority if it did not require any "server-side" code, if somehow, resources were true resources and there would never be a bit of logic in between the client and the resource, only GETs, and PUTs. But we all know how much client and server code we have to write between a button click and a resource. If Stefan can show me how the goop between the client and the resource magically disappears, I will become a RESTifarian.
11/11/07 :: [SOA] The Dark Side of REST [permalink]
I start seeing a pattern in the REST community. It started when I watched Stefan's presentation which made me uneasy. Today, I found that Subbu quoted me in his last post:
"REST does not support Web Services Choreography, and hence REST is unfit for the enterprise"
He used quotes ! as if this was my argument. Again my argument is that REST seem be an interesting technology for Presentation composition/federation (I would argue because it is lightweight compared to WS-*) but it is unfit for process and information composition/federation.
I'd be happy to think this is only my imagination. But frankly, if this is the kind of things you are ready to do to respond to arguments, I start seeing the REST community as possibly being worse than your typical BigSoftwareVendor sales rep. At least they have a boss they can blame for the red tape.
Isn't this is the old Scare & Blame scam? The kinds of arguments they use doesn't matter, right? You must trust the REST community since they scare the hell out of you and they found who you can blame for it. Don't you?
11/10/07 :: [SOA] I think Bill and I somehow agree [permalink]
I don't have too much time to continue this debate, I think there is hope that the RESTifarians will leave the (WS-*+REST) crowd alone.
If you read Assaf's comment on Bill's post:
The orchestration landscape is a mine field of different opinions. For some, it's all about top-down static endpoints using infrastructure to exchange control messages in the background. For others, well at least me, it's about message passing between processes, and REST fits nicely in that.
This is what the RESTifarians really care about "message passing", this is why they are so prompt to ignore application semantics. They tell you, all you need is send and receive messages (they don't really care about resources (see this post), a trick to avoid using a contract) and they are correct, you can do that efficiently with REST if you don't need security, reliability and transaction.
This is what Bill replied:
This is the same type of discussion I had with Nick Malick. Just like Nick, the RESTifarians come up with arguments that only hold in LaLaLand and the rest of us have to deal with the consequences. We, in the trenches, have to justify our choices (more complex and more expensive) to our employers and co-workers because someone wants to rule the world"
Wow. Jean-Jacques has more or less described how I've felt about commercial WS-* technology for years, and better than I ever could. There's a great irony here that we shouldn't lose sight of :) I really would like to meet up some day.
I am glad you understand the pain. I don't see crazy pitches at work but we have to acknowledge that in any organization (IT or not) there are people that are ready to shoot things they don't like or they are afraid of. The more outrageous the comment the more it circulatesand ultimately muddy the waters for every one. Whether it is Nick, Joe and Dave Linthicum on Enterprise SOA, whether it is Dave Chappell on SCA, Tim Bray on WS-* or some of the RESTifarians on REST. I respect all these people, this is why I am pointing out some language that I find excessive. Richard thought that I was trying to apply censorship to everyone. No this is not true. I am just reflecting on an architect part of a complex SOA initiative and receiving these posts via email and having to defend his/her choices in front of his boss and colleagues (I did receive Tim Bray's comment via email and had to explain why he said what he said).
I don't claim I hold the truth but I care to be truthful in what I say. It does not mean that I am immune to saying something incorrect, but if this is the case, I would be the first one to admit it. We have to acknowledge that this field is very complex and nobody wins in playing bowling with words.
I am ready to put this discussion behind I think all have been said. As I said I have enough argument to answer any question on REST vs WS-*. My position is clear, apply REST (presentation federation) where you can and WS-* where you cannot (process and information federation).
Bill, if you ever come to Seattle, let me know.
11/09/07 :: [SOA] Bill de Hora Responded to my REST Posts [permalink]
Bill responded to my post. Thanks BIll for putting so much effort but I won't respond in detail because we will never convince each other but I can convince my manager with what I wrote. I don't think I said things that were different than what Sanjiva said at QCon, though I can easily admit he did it a 100 times better.
I think he summarizes well the discussion between RESTifarians and the REST of the real-world:
I have to wonder about this kind of criticism - it's like criticizing a car for not being a boat - but why are you looking at cars?
Well the reason why we have to go through this kind of discussions is precisely because we need boats and the RESTifarians are trying to sell us cars for every transportation need. For them buses, airplanes and boats are useless since we have cars. When they will hit the water, they'll build a bridge. The REST of us are simply requesting to be left alone.
This is the same type of discussion I had with Nick Malick. Just like Nick, the RESTifarians come up with arguments that only hold in LaLaLand and the rest of us have to deal with the consequences. We, in the trenches, have to justify our choices (more complex and more expensive) to our employers and co-workers because someone wants to rule the world with a sub-optimal technology. Look at this post from Stefan taking notes on Steve Vinoski conference at QCon. No offense Steve, but if Stefan took notes from what you said, you sound just as good as an ESB salesman.
- REST is hypermedia, but not only suitable for browsers
- major difference to SOA: certain constraints to achieve desirable properties
- performance, scalability, portability, simplicity, visibility (ability to monitor and mediate)
- modifiability, extensibility, evolvability, reliability (handling failure and partial failure), failover, load balancing
- SOA guy: everybody wants those properties, including all SOA systems
As SOA has no constraints, it doesn’t help you there
This my version of an ESB salesman:
- ESB is middleware, but not only suitable for EAI
- can do SOA too with these ESB capabilities:
- performance, scalability, portability, simplicity, visibility (ability to monitor and mediate)
- modifiability, extensibility, evolvability, reliability (handling failure and partial failure), failover, load balancing
- SOA guy: everybody wants those properties, including all SOA systems
Bill's answer shows that we live in a complete different world. RESTifarians argue that :
a) if your favorite technology doesn't do what you need, you call a friend to write a post or a thesis and voila, you have a standard that everyone can use
b) modern application semantics are not necessary, constraints are enough
If you want to see the real mess that REST is going to get you into, please take a look at AWS. If you are Amazon or Google you can deal with this mess, if you are an IT organization you can't. IT is dying of all the hacks and tricks that have been invented over the past 15 years to fix an application model that is simply broken. For once we have application semantics (including REST) that enable us to construct composite software from reusable assets. RESTifarian are denying our rights to use what we think is the right direction for IT.
The funny thing is that even though you say (and think) that REST is great and should be used where applicable, this is not enough for these people. They want complete victory, eradicate the world from all form of middleware, HTTP rule. They can't even listen to themselves anymore. They are trying to sell you the "scalability' of the Web when hardly anyone need to build systems like this (except Amazon, Google and Yahoo). Is that a scam guys? are you playing the tail-head thing? are you trying to grab the tail and you will add the semantics as needed those very same semantics that are available today often at no cost? If this is your plan, I am disgusted.
11/07/07 :: [SOA] Yet, Another REST Fallacy [permalink]
Tonight, I was reading Mark Masterson's blog where he continues his thoughts on an enterprise 2.1 manifesto. One element of the manifesto favors: "Resource orientation over service or object orientation for distributed systems design". I followed the link he referenced to support his assertion and I came across this article from Alex Bunardzic (August 2006). This article seem to have been blessed by Stefan (based on a comment he posted). So I am assuming he considers this is REST compliant.
It is always interesting to play with architecture concept. This is an activity that should be widely encouraged and practiced. Recently I came up with a "Feed Oriented Architecture" concept, but I did not claim to displace anyone, Resource Oriented Architecture or Service Oriented Architecture for instance. Playing, opens your mind, encourages you to explore without pressure whether you are a software architect or say a residential architect.
One day however you have to build something real. So what's real for Alex? The paragraph that caught my eye in Alex's musings was this one:
Once the representation of the resource’s state has been consumed, the requesting client may make a decision to modify that state. For example, if the state of the identified tennis court is that it is free on Saturday morning at 10:00 am, the client may decide to change that state to booked. The client sends another request to the resource, this time using a different aspect of the protocol (basically, using the aspect number 2). The resource handles this request, and, obeying its business logic, makes the transition from its original state (free) to its new state (booked).
Alex claims that resources have states and of course there are transitions between these states. So far so good, that's an established concept. The plot thickens when Alex tells us the "client" may decide (huh...how?) to change the state of the resource. Well the "how" is by sending a request of type 2 (Alex explains: "Each resource knows how to make a transition from one state to another state", what "knows" means? exposes a method?). Once the resource receives the request is executes its logic and make the transition.
So let's replay very slowly what Alex is telling us:
- when he says the "client decides" he means the "user" in the browser, looking at the resource representation, making sense of it (based on a common language, say English and common understanding of a link semantic) and pushing the link to reserve the court.
- How does the resource knows who is reserving? well that's a mystery
(I know I can hack it in, I am talking about achieving it in an
interoperable way).
Now let's look at how a "server" would interact with the resource (not a user). How would the server "know" that this link would reserve the court? Are we assuming a natural language capability embedded in REST? Well in reality, this is probably the untold constraint of REST. This is possible, I did built a natural language engine in 1996 to interact with "objects" using Objective-C (and NXDictionaries to support synonyms of nouns and verbs) at Hughes Research Labs. This was the best piece of code I ever wrote and it looked quite spectacular. I remember a Stanford graduate looking at my experiment and yes, I could interact with it in "natural language". He sat down at the keyboard and started to interact with the experiment too.
I digress, huh.. wouldn't a contract be necessary to establish the semantics of the action? How would the server know otherwise?
I'll leave it there, REST is not viable as a style to support the architecture of enterprise information systems and deliver Presentation, Process and Information federation. I can see a world where REST owns say 80% of the Presentation federation. The other 20% is owned by WS-HumanTask and WS-RemotePortlets. I don't see REST achieving process and information federation in the next 10 years (that's how long it took for WS-* to mature)
Sincerely I am surprised that Stefan considers it to be REST compliant, but it does not change my argument, the real question is how do you model an action to transition a resource from one state to another. The answer is that you don't.
Now I think the burden is on the REST community to explain their plans to support the WS-* application semantic such as Orchestration, Peer-to-Peer, Events, Intermediaries (including pub/sub), Security, State Alignment which are going to be extremely hard to build into the model without doing a copy cat of the WS-* specs.
Good luck with your REST enterprise initiative.
11/07/07 :: [SOA] The rEsT FALLACIES [permalink]
Mashups are a great concept (check mine), when you build an enterprise class solution you really have to mashup (federate) GUIs, processes and information. You would be passing on tremendous value if you would never use process or information federation.
IMHO, this pictures shows how much a loosing value proposition REST is. As Stefan pointed in his presentation introduction, REST is good if you are creating services (huh?) that need to scale massively in terms of consumers. I can appreciate why Google, Yahoo and Amazon love REST. I can see how REST services delivering web parts as resource representation can be so convenient to use and therefore why REST is here to stay.
However, when you have "real-world" problems, that do not require the kind of scalability that Google is looking for, then you WILL be confronted to the need to express:
- intermediaries: to help you perform security checks, routing or transformation at wire speed. You are not even on your own there, REST can't do it.
- interoperable reliability: REST has tricks, but not a spec. How to you do message ordering in REST? well you don't, you are on your own.
- contracts: how can you do versioning in REST? you are on your own. So if all you are doing is displaying an HTML fragment on your page, you don't care so much about versioning. You don't really care too much about the structure of what you are receiving, but in general, in the real world, you care about knowing when the "contract" has changed. You also care that the contract be machine readable so that you don't have to "implement" a service proxy on the client side, the proxy is generated for you to use within your business logic. What's a contract for REST? you have one contract per resource? no of course, per resource type? but there is no concept of resource type in REST. You are on your own for contracts.
- peer-to-peer: Roy fielding himself say REST can't do peer-to-peer it has to be "relaxed". You are not even on your own there, REST can't do it.
- Dependency of Injection: DofI of resources? no, not really, since REST does not have the concept of a service, you can't do DofI, you can't arbitrarily replace the root of URIs, on the fly. You are not even on your own there, REST can't do it.
- state alignment: what most people don't understand is that a transaction aims at aligning the state of two or more software agents. Meaning, at the end, each agent is in a state that is known without ambiguity by all the other agents (whether it failed or succeeded). To achieve state alignment, you need a state alignment protocol. This protocol is one of the hardest thing to design (talk to Eric Newcomer or Mark Little). You can't do one per project, all the time, I bet most of your developers (including me) wouldn't be able to design one. These people had years of discussion on this topic. Doing that own your own as you need it?
- pub / sub: that's an obvious one, REST simply forbids pub/sub all the way. You are not even on your own.
- Orchestration: the REST camp, with people such as Steve W. says, I can do with a little script. You simply have no idea what you are talking about. On average an orchestration reduces the amount of code by a factor of 100. Orchestration style code is hard to write, it is extremely stateful, need to handle correlation and concepts such as hydration and dehydration. REST can't even do correlation because it does not have a contract. There are no XSD associated to a resource, a resource is only defined by its URI. What a warm fuzzy feeling to be on your own to deal with correlation and dehydration/hydration.
- Resource update: it is completely naive to think that resource update can easily be modeled in REST. Since what you get on the client is a "resource representation" which most often is a partial view of the resource itself (and different clients could get different representation, not just from a format perspective, but from a structure perspective), how do you communicate updates back to the resource? you are on your own. SDO offers a "change summary" that clearly communicate what the client did to the resource representation. The client also gets instant feedback if it's trying to do an update that can be invalidated by contract. An update can of course still fail on the server after some business logic is applied. If you want to be able to do client side contract based validation, you are on your own.
- Choreography: this is probably too abstract for people to understand when you need it, so I'll let you get away with this one, but you are still on your own.
The REST fallacies are simply outrageous. The bottom line is that REST is a terrible web service model for the enterprise. Some people are trying to set you back ten years when it comes to building enterprise class (not Google class) distributed information systems. A lot of companies would like to have to deal with Google scale problems, but the reality is that they don't and will never need to do so. On the other hand they will deal countless times with process and information federation, not just mashups.
I'll repeat what I have said, I am not anti-REST, REST principles are great, REST style makes sense and you should apply it wherever your can. I am anti WS-Bashing. Both WS-* and REST have room to grow and make sense. It makes no sense to oppose them: they need to learn from each other. I don't think it makes sense to have them converge today either, maybe in two or three years we'll revisit the question.
Good luck on your REST enterprise project !
11/04/07 :: [SOA] Stefan on REST [permalink]
Another group that wants to kill SOA is some of the REST folks. Stefan posted a pointer to one of his REST presentation. Since you don't see the slides in the video you may want to GET them ahead of time.
Ok, so I said, I won't participate in a REST debate again, I apologize, it is just that I respect Stefan a lot and at the same time I was very uneasy about his presentation. I read it as yet another attempt to kill Web Services and with it SOA, because if all we had was REST we would be unable to build a Service Oriented Architecture. As a starter, I'd like to disclose, I like REST, it's brilliant, surely enough the web would be very different if it had been designed on OO/Corba principles.
In his presentation, Stefan makes a very good point about resources vs services, REST sees the world as "un-gated" resources, while services are the gate to a parking lot full of resources. I fully support this view and this argument, but is it enough?
I start to part from him on the "application semantics". He ends his talk by saying "HTTP is good enough", meaning that all you'll ever need to do to construct a distributed application, you can do it with HTTP. I must admit I am an application semantics freak. I don't think people should be coding their breath away, I do think that application semantics should be defined to write as little code as possible (this does not mean zero code). I come from a NeXT background, that might explain why.
To illustrate how HTTP application semantics were enough, Stefan sent me a link that describes a "stock trader" application. For sure, the paper is full of "additional" semantics not directly supported by HTTP (in other words you have to code them in, each time) such as reliable messaging semantics or "operation hacking". The Reliable Messaging solution that is provided assume that the server talks to a "client".
Operation Hacking consist in mapping "business operations" to URI, here is a sample from the article:
| URI | Method | Collection | Operation | Business Operation |
|---|---|---|---|---|
| /acct/ | POST | accounts | create | register |
| /acct/{acct_id} | GET | accounts | retrieve | getAccountData |
| /acct/{acct_id};profile | GET | profile | retrieve | getAccountProfileData |
Note, that I am making fun of this but I do believe that this is a great design pattern for SOA. I am making fun of it, because REST claims its opposition to SOA (resources first, smash the services -the ugly services lock the poor resources behind doors). But, how do I read the URI? Well there is an /acct service which has a getAccountByID operation,... because at the end of the day, this is really how you will end up implementing it, with a service (the system of record for this resource type), not one resource at a time. What Stefan is saying though, is that why do you need WSDL and WS-* when I can give you a URI and without anything else you can interact with the resource behind. Just like WSDL you don't need binaries, but since your a