ABOUT | COPYRIGHT

Service Oriented, Process Centric, Model Driven Heresies

	del.icio.us
	Digg
	DZone
	Furl
	Reddit

Subscribe in a reader

09/16/08 :: [REST] Wag the Dog [permalink]

Everyone who knows me, knows that I am not a WS-* guy. So it is a bit odd to take the stand and defend a technology that I find attractive only because we can fix it and we should not spend the energy to reinvent the wheel. As an introduction to this second response to Stu, I would like to state a strong belief. I would like to state the belief that our industry has delivered all but sloppy technologies over the last 15 years: they barely help you do your job, and vastly useless or buggy or both until version 5.0, without a clear upgrade path in between and subject to change any day, even though they were never built for change. I don't want to overly criticize anyone here, this is probably a side effect of a very innovative space. I am pretty sure that people in BioTech could say the same. I just would like to see people reflect on the effect of an ever changing sloppy landscape on people building and maintaining solutions.

Since Tim has twitted my previous post, he might twitt this one too, so I might give a bit of background. In this post, I will summarize all the discussions I have had with the RESTifarians and demonstrate that they offer nothing different than what others have offered over the last 15 years. However, our industry (and our economy) is at a very different point in its lifecycle than it was 15 years ago. The question I would like to answer is what's going to happen if we Wag the Dog one more time in this context? What I mean by Wag the Dog is "paint a pretty picture" of a technology that end up being just "another sloppy, useless, buggy, without a clear upgrade path" technology? Can IT withstand it? can the business withstand it or have any patience for it?

The thesis that I will defend here is that technologies such as WS-*, SCA, BPEL... are good enough, they can be fixed, made more useful, less buggy and stable to the point where they will enable sustainable upgrade paths. I will demonstrate that the RESTafarians proposal amounts to starting over, to create the exact same thing wasting years in the process and creating possibly a final blow to IT, loosing any and all credibility from the business.

1. What is the end goal?

I don't want to sound to enterprisey here, by the end goal today to create the programming model of "connected systems". A connected system is built from reusable (autonomous) assets, assembled into a solution. These assets can live within or outside the firewall, within the control of the enterprise or a 3rd party. The key concepts behind "connected systems" are federation/assemblies and composition.

2. What are the contenders for creating connected systems?

RESTful HTTP: resource oriented programming model based on the REST architecture style
MVC: action oriented programming model
WS-*/SCA: service oriented programming model
EDA: event-oriented programming model (this is not well defined, please see the great work of Udi Dahan for an example)

There is probably a few more, but let's assume that we have the 4 main ones. Each of these programming model rely on endpoints to implement federation/assembly and composition concepts. As I mentioned in a previous post, they all represent perfectly valid solutions to some classes of problems. The question is, can one of them make all the other obsolete? My answer is no. I could argue that the weakest of all is RESTful HTTP, let's see if I can demonstrate that. None of the other programming models claim an hegemony over connected systems.

3. So where does RESTful HTTP comes from?

Roy Fielding who contributed significantly to the design of the Web standards such as HTTP wrote a Ph.D. thesis explaining a posteriori why the architecture of the Web contributed to the scale that we experience today. I personally believe every word of his thesis. Roy has done a remarkable job both before and after his standard work. But Roy was also very clear, he designed HTTP and REST based on Tim Berners-Lee's requirements:

What was needed was a way for people to store and structure their own information, whether permanent or ephemeral in nature, such that it could be usable by themselves and others, and to be able to reference and structure the information stored by others so that it would not be necessary for everyone to keep and maintain local copies.

Tim was a scientist at CERN and information is critical to scientists. I can't imagine what it is to be a Ph.D. student today when you can google anything. I remember when I first entered the physics library at Penn State University. All the journals were there, decades at a time. My search engine then, was based on table of content and indexes. When I was in a Ph.D. student in France, we did not even have anything like that and I had to rely on a librarian to help me find and order articles relevant to my research. Well, we also had "current contents" (the blogs at the time). Anyways, the Web was never created with the Enterprise, B2B or even connected systems in mind. The Web was designed as a very basic content management system (without full text search because of obvious federation problems). Even though it was designed to PUT and DELETE some new and updated content, these features were never really opened to the public for yet obvious security reasons. Actually they may not have been so obvious after all when we look at the popularity of wikis. So let's assume that HTTP, the Web and REST were all designed on the foundation of creating a massive, federated content management system. Boy, was this mission accomplished. That makes Tim no less than a modern Gutenberg and his influence on the world is no less than the one of Johannes.

But, can a technology that was invented to fulfill the requirements shown above really work beyond what it was designed for? as-is? This is no less than the claim of the RESTafarians. They are telling us that all the capabilities delivered by RESTful HTTP are enough to build enterprise b2b capable connected systems. Is this another instance of Dog Wagging or is it true?

4. REST in depth (from the perspective of a connected system)

4.1. Connected Agent Boundaries

REST offers two types of boundaries:

the resource
the network authority upon which a particular resource depends on

In REST, and RESTful HTTP, you interact with a resource and a resource is located within the boundaries of a network authority (a domain or subdomain).

This really does not help much security, monitoring and management as each resource can actually be implemented or stored on a different server. REST does not support the definition of arbitrary boundaries, it is not even a question of defining an interface other than the RESTful uniform interface, it is really a question of defining "things" that you can manage, monitor and secure, things that you know when they are down and which impact they have on your business. Again, Tim's vision of the Web would have never required anything like that (I won't repeat it each time).

=> boundaries are inadequate

Actually this boundary scheme also create an insurmountable problem in the programming model as all resources that participate in the same RESTful connected system need to belong to be dependent on the same network authority. If you need to post a PO to a customer and both the PO and customer depend on separate network authorities, you are out of luck and you are on your own to stitch the references together. We know how practical that can be in the SaaS/Web 2.0 era.

=> boundaries are vastly inadequate

4.2. Identity

RESTful HTTP is kind of unique compared to the other programming models in the sense that it supports the concept of "resource", unfortunately, REST does not support any notion of "identity", in other words I cannot rely on URIs as an identity. U means uniform, not unique. Two URIs can point to the same resource and I could return different resources for the same URI (think of a quote system where I get /quote/{symbol}/last), because it happens that you can never prevent someone to express an operation (e.g. getLastQuote) with a URI syntax.

This means that if you want to use the resources in the resource oriented programming model, you have to create an identity mechanism like the other programming models. This model has to be federated of course. I also bet that all programming models could use the same federated identity mechanism as resource orientation can be layered on top of service orientation, an action oriented and an event driven programming model.

=> resource identity is not available in REST

4.3. Query language

If you give a URI syntax to people they can find all kinds of ways to abuse it. Their favorite abuse is to create a query language, nothing is easier, unfortunately, they are also proprietary and they are unrelated to the media types they apply to.

=> proprietary query languages will become the norm

The most hilarious part of REST, IMHO, is the need for APP crutches to be able to deal with collections. In other words, REST itself cannot return collections of objects. The Web was designed as a basic content management system in mind, and a table of content would be enough to organize a collection of documents. REST has no provision for collection of records. Unfortunately in the enterprise collections are kind of the norm. So what do you do? You call Joe and voilà, you get Atom Publishing Protocol (APP) and you have a standard format for returning collections. Of course APP is RESTful. So, so long the idea that "all you need is REST". The truth is "you will need REST-*".

=> RESTful HTTP can't deal with collections without APP

Links are no relationship (as in ER) either. Links are unidirectional when data models rely on relationships which are bidirectional associations.

4.4. The Uniform Interface

Ah, the uniform interface, my favorite topic. I think nobody believes anymore that the interface to a resource can be uniform, except maybe for Steve. Yes, if the resource is a web page, its interface is by definition uniform, all web pages have the same. If your resource is a customer, a PO or an invoice, well they have actions associated to them. You can actually encode actions in a resource oriented way. This is a bit twisted and this was best explained by Obie Fernandez:

If you want to invoke a login action (MVC), you post a "session" resource to the user resource that wants to login.

The only catch to the problem is how do you figure out the state the resource is in? Do you

get the /user/{id}/sessions/last ?
search the /user/{id}/sessions collection returned in a beautiful feed format?
get it from a field in the /user/{id} resource?

The question is more important than you might think though Roy's REST does not give any guidelines for implementing actions, states and state queries, IMHO, the are no true RESTful way to implement an action, some states and the corresponding state queries. 1) is an operation and creates identity side effects if you use it as a URI to a resource. IMHO, this type of URI should be forbidden. 2) is the closest as it does not induce side effects, but collections are not part of Roy's REST and you have to "search" for a session that is open. 3) introduces an awful coupling, as you start CRUDing your resources. Eventually your developers will be tired to add secondary resources (e.g. session) each time they want to invoke an action, and for convenience, most people will end up updating the user resource with a PUT and GETing it back. If you do that you are CRUDing and you are creating a terrible coupling between the resource consumer and the resource. This should be strictly forbidden.

=> too many ways to encode an action semantic, no clear way to get the state a resource is in, most people will choose a CRUD approach.

REST's uniform interface actually induces a lot more terrible side effects:

The biggest one is Versioning. Versioning cannot be implemented in REST. Pete Williams brought back the REST community to a point of sanity by suggesting using Resource Media Types can be used as a versioning scheme (before that the best practise was to encode the version in the URI :-)). Practically, when you "GET" a resource you'll have to specify the Resource Media Type + the version of the type you are looking for. In theory media types are supposed to be standard (IANA) so adding a version number to it, is not really practical. Unfortunately since a rest URI couples access with endpoint, it means that all the code for all the versions have to be implemented behind the resource URI or you have to proxy every resource. By contrast because WS-* or EDA exposes unique end points a consumer we have the choice to direct a consumer to the right version or implement forward compatible versions without disrupting the operation of older consumers.

Furthermore, because of the boundaries that REST provide, there is really nothing to version. You rarely version a resource type alone. By contrast service orientation provides a unit of versioning that is a lot more meaningful (and not just for versioning, for operations, management, security...)

=> REST was never designed to version and can't version

The other problem with Pete Williams's suggestion (other than it is incomplete because of the boundary problem), if you use Media Types for versioning you can't really use them for anything else because the combinatory problem is going to become quickly overwhelming. Let's assume you want to also express some access control logic in the media type (who can access what), the only way to do that is again with the Media Type. If you combine: the "true" media type + version + access control, how many individual media types are you going to need? The thing that does not work in REST is that there is no envelope. The RESTafarians carefully crafted the APP crutches to deal with collections because they had no choice, but they cannot settle for any other kind of envelope that is not user readable. If they loose the browser then their programming model has absolutely no value whatsoever.

The claim to fame of REST is the way the uniform interface and hypermedia (Hypermedia is defined by Roy as the presence of application control information embedded within, or as a layer above, the presentation of information) work together. This is truly a marvel of engineering. Unfortunately it works only when a user is in the loop -above the presentation of information. A user can make sense of any changes in the actions embedded in the resource representation. Unfortunately a software agent can generally not do that. Because die hard RESTafarians refuse to define a contract, then any modification on the server cannot be validated by the client. So the claim you hear most often by the RESTifarian that the uniform interface helps you make changes on the server without impacting the client is only true when a human is in the loop. In a connected system this claim is a complete Wag, probably the biggest wag of all and despite all the discussions we could have had, people like John Heintz keep wagging this absolute fallacy.

=> REST cannot deal efficiently with modifications on the resource side (the action interface) when the consumer is not a human

The uniform interface is unidirectional and synchronous. Best practices of connected system construction require a bidirectional interface representing interactions between resources and asynchrony.

=> REST is missing fundamental connected system interaction patterns

The uniform interface cannot support events

=> REST's uniform interface was never design to create notifications of particular occurrences of a state

Last but not least, one of the biggest problem of resource orientation in the enterprise is that none of the legacy systems are resource oriented. If you layer a resource oriented access on top of these legacy systems (the resources have to live somewhere right?) then you are going to experience a terrible mismatch between this resource oriented access and the way these systems operate. O/R mapping is going to look simple in comparison. You can trade this mismatch for mashup flexibility for instance, but mashing up is only a small part of connected systems.

=> Resource orientation does not work well with legacy systems

4.5. Security

REST does not offer any federated security mechanism, including no support for user identity propagation which is essential in Enterprise system construction. You need an envelope mechanism to support identity propagation and REST is using APP (a non extensible envelope mechanism).

=> REST security infrastructure is vastly incomplete to support connected systems

I'll let REST get away with Transaction and Reliability but they can't support that either.

4.6 Caching

Lots of RESTafarians are prompt to point that REST enables caching. Yes, the Web needs caching and offers lots of opportunities to use caching. In the enterprise the story is far less compelling. Entity Beans tried to deliver a caching engine to enterprise data without any success. Enterprise data simply does not cache well.

=> Caching is useless for enterprise data

4.7. REST from the perspective of BPM

REST programming model is incompatible with BPM. User activities need to be implemented with a proprietary mechanism and because of it's lack of bidirectional interfaces REST cannot support events or orchestration components used for instance to manage the resource lifecycle in a business process.

=> REST is going to set BPM 10 years back

5. Where do we go from here?

The conclusion is that REST has only two dimensions to express application semantics: URIs and Media Types, this is not enough. Whatever tricks the RESTafarians will come up with, they will be never enough room there.

So, I hope everyone is convinced by now that the RESTafarians are wagging the (IT) dog by what's left of the tail. Their claims are far stretched if not plain wrong. Nothing that I said in the "REST fallacies" post proved to be wrong:

Yes, I agree there are a lot of interesting concepts in REST that can be reused when building connected systems, but at the end of the day, REST alone cannot drive IT where it needs to be (nor WS-* or SCA or EDA). IT needs a programming model where resources, processes, events, (inter)actions and services are exposed as primary concepts rather than reified behind a single one. The era of "Everything is a XXX" is gone (where XXX is: resource, process, service, event...). This approach has been a dead end each time someone came out with such monolithic programming model. You just can't reify some concepts behind others, just because you are Turing complete. Turing is no match for this kind of job (metamodel driven programming). I have created wsper to show how these concepts can fit in a programming model.

So, Stu, it is probably time to move forward, none of these arguments will change until you create REST-*. I hope that you will agree with me that, indeed, the fundamentals of IT are not strong enough to sustain another "wagging the dog" wave. Maybe we could agree that delivering "sloppy technologies, that barely help you do your job, and vastly useless or buggy or both until version 5.0, without a clear upgrade path" is the reason why IT cannot do its job. The (other) REST is no different. Nice try guys, you say "look this works for the Web, it is well-formed, stable, not buggy, useful", but this is only true in the Web. In the Enterprise your statement is fallacious and frankly ludicrous, you will have to recreate REST-* and REST-* is sloppy to say the least.

Steve, in the most ironic fashion, REST could prove to be a disruptive technology, but in the way that was never predicted by Clayton Christensen, REST, as yet another Wag the Dog technology is about to transform consumers into non-consumers, because you see, if IT can't deliver value, it won't get any money and be pair down to keep the light on, giving its true meaning to "IT as a utility".

Stu, you may also agree that at this point creating communities that conflict with each other is not going to solve any problem. We need a Willy Brandt that will reach out across the iron curtain and stop this nonsense.

Hopefully architects, like me, have now enough material to go back to their management and put the (other) REST to rest.

Jean-Jacques Dubray, September 16, 2008