del.icio.us del.icio.us
Digg Digg
DZone DZone
Furl Furl
Reddit Reddit





 Subscribe in a reader



03/18/10 :: [REST] REST Versioning [permalink]

I was disappointed -to say the least- by Stu's response on REST versioning. I was expecting a lot more. I am not sure what to say about that:

What is missing, on the other hand, is a general purpose agent programming model that allows the easy expression and construction of hypermedia-consuming, goal-driven applications.

Let's be polite and argue that there are many simpler problems we need to solve before we tackle that one. So, allow me to write the response I expected from him.

First, I would have like to see Stu's model of REST. I am not sure you can just stumble upon the kind of programming model he is talking about. It has to be "designed". So, since Stu understands actions and resource lifecycles, he could have at least articulated how they fit in REST, since no one is talking about the "uniform interface" any longer, it would have been timely to establish that once and for all. It would for sure limit the amount of CRUD being generated.

So, allow me to offer, what I think REST could have been (actually is):

There are 3 RESTs out there. Roy's REST or REST Core. That's not very useful by itself as a programming model. Then there is the practical REST which adds queries (with properties) and collections. That's the CRUD of REST. The "Full REST" has actions and events. How do actions relate to a resource? Their invocation triggers a transition from one state of the resource to the next. These concepts have been on the table since the 60s. But in 2010, we are still debating about them. WOA ! That is called progress.

How are actions expressed in the full REST? Quite simply, RESTafarians do that every day: you just transform a verb into POST + noun. For instance you want to pay a bill, you POST a payment to the Bill resource. Yeap, that easy, in French this is called "encoding". I am glad ThoughtWorks charges you $300 an hour to teach you this kind of thing, this is as close as printing money as it can be.

So, in case you still wonder, no, an interface to a resource is not uniform, far from it -even Jim and Savas finally got that after 3 painful years-, no, CRUD is not the way to go, and no, people that tell you "Data Services work", have no idea what they are talking about. Sorry. If it is not clear to everyone, a CRUD based programming model forces you to push "reusable" business logic on the client of the resource, creating large levels of redundancy, not to mention that hardly anyone will let the state of their resources be decided on the client. For those of you that still don't know what REST means, it stands for REpresentational State Transfer, not for REsource Data Access. A representation of the state of the resource is passed to the client which indicates the actions he or she can take. Brilliant, Roy's REST is brilliant, he never claimed that what you needed was a bunch of CRUD.

That's why Actions work so well with hypermedia, actions are embedded in resource representations, this actually happens billions of times a day on traditional web pages. But mysteriously, the RESTafarians did not notice it, unless they don't want to "talk" about it (Stu?). I mean in the Cloud even Tim Bray finally figured out that indeed there were actions like "start or stop a server", how about "reboot a server". He even figured out, by himself, that you can't CRUD those actions.

This little action "secret" is an inconvenient truth because the RESTafarians precisely sold you on the idea that "contracts" did not exist: all this XML Schema and WSDL crap was just here to jack up the price of consulting engagements and that they will save the world by ridding it of these pesky contracts. If that is not boloney, what is Stu? But I digress. Let's go back to the main problem: REST Versioning.

Before I continue on, I want to reiterate why versioning is so important in connected systems. It is really important because you can't predict the future. Yes, you can't build an asset today with the expectation that in 3 years someone will come around and consume that asset as is. It might happen less than 1% of the time. Don't waste your time designing for that or Dino Buzzati will end up writing a book about it. So, how do you go about versioning in a connected system? It's quite simple. It is not the new consumers who "reuse" the old service. It is the old consumers who can still operate with the "new version of the service". Reuse happens the other way around. This is what is called "Forwards Compatible Versioning" because it is different from "Backwards Compatible". Backwards compatibility happens when you change the consumer and it can still talk to the old service. Versioning is the reason why RPC failed, this where CORBA failed, this is where JEE failed. This is where Spring fails. Versioning is actually such an acute problem even in monolithic programming models that people had to invent new technologies like OSGi to deal with it in  OO. I am glad that people like Dave Chappell are paid thousands of dollars per hour to claim that "SOA is a failure", that "Data Services are the only things that work" and all that because you can't predict the future.. What a gig.

Kjell and I wrote an article on how "Forwards Versioning" works in Web Services. I would recommend you take a look at it if you are not familiar with the concept.

So, now that we have a full model of REST, what can be versioned and how does it work? How different is it from Web Services?

Well, there is a lot that can change in a REST-based information system. Let's review the use cases one by one.

Use Cases Variants Solutions
Network authority Network authority is gone If the network authority is gone, you are out of luck, however, if it is still there, it can express to resource consumers that the resource can be found somewhere else (HTTP code 301 Moved Permanently). This is a bit more work than changing the endpoint of a Web Service, but it works. In particular the client can adapt to the change without configuration like in Web Services.
Resource has moved to a new network authority
Resource New resource added to the network authority Since REST encourages the use of Resource Representations, there is consequence, as long as the changes do not impact the resource representations. When a new resource is added to the Network Authority, there is no interference with existing ones.
The resource structure has changed
Query Add new queries REST makes it easy to add new queries to an existing resource. There is no penalty for "adding" a query. Existing Consumers are not impacted.

Operations can be added easily in WSDL, without communicating the changes to the new client, it is just as easy.

When you change an existing query, you want to make sure that your changes remain compatible. That's fairly easy to do, both in REST and Web Services.
Change existing queries
New State New state implies new transition That's a change in the resource structure. So the same comments apply.

Removing states is trickier because it generally implies that some actions can no longer be invoked.
New transition   Well, that's a new action, so a new POST+noun. A new sub-resource need to be added to the main resource. Just like in Web Services, you can add more operations without impacting existing consumers as long as the new version of the state machine is compatible with the old one. Old consumers will simply be unaware of the new transitions and states.
New Media Type   Changes in media types can be made forwards compatible courtesy of XML Schema, just like for their WS-* counterpart.


Check, Check, Check, just like Stu predicted, Web Services forwards compatible versioning foundation came from the W3C so it works for REST as well. It actually works even better than Web Services because you don't even waste time updating a contract. You just create a new URI syntax, you email it to the new consumers and voila.

I forgot something? What did I forget? Ah, a small detail. You remember, I keep saying "REST couples access and identity". Such a small little detail. It means that the business logic behind a GET ResourceRepresentation or a POST Noun is bolted to the URI. So when I say "GET /customers/123" I am bolted to a piece of code annotated with the innovative JAX-RS notations. That's a problem because /customers/123, as Stu explains it, is also an identity, a foreign key to somebody else. So I can't change it (remember cool URIs don't change). How do I go around that? Pete Williams provided a work around using media types. Usually Media types are used for content negotiation, he suggests to create a media type per content type, per version. It works. We know how well people are going to keep track of their media types and versions.  Just try to submit these types to the IANA ...

Does that work for POST + noun? It could. But, it is also safe to use a URI syntax because a POST of a resource to /customers/123/payments is actually an operation of a service end point, REST allows the server to append the payments in a different locations, so it would be quite ok to POST at /customers/123/payments/v1 and at /customers/123/payments/v2. The payments though would all be under /customers/123/payments/

So yes, versioning kind of works. Are we done? No, there is still another detail missing: the unit of versioning. Yes "real-world" resources have complex lifecycles and often composite states. Each composite state/lifecycle corresponds to a unit of versioning. Unfortunately in REST, there is no "unit of versioning". As you have seen, you add stuff here and there and before you know it you lost track completely of what exactly you are versioning. The key problem is that your business logic for a single resource could potentially be dangling off dozens of endpoints. So you have to painfully manage all these annotations in a forwards compatible way. I am not sure that is less work than creating a WSDL. In REST, there is not even a trace of the beginning of a "unit of versioning" beyond the URI itself. I will let you compare that with the elegance of versioning with Web Services contracts where you can replace the the business logic layer at the change of an endpoint and assemble different WSDLs for the same service and different consumers based on what you want them to understand. As a matter of fact the key design pattern in SOA is the separation between the interface and the implementation? So what do the RESTafarians argue about? They want you to bolt the interface to the implementation. They even create annotations to make sure you'll never set free from this architectural atrocity. It's like designing a house with the bathroom behind the front door. RPC, CORBA, JEE all bolted the interface to the implementation. Somehow, Web Services escaped that fate. But the old CORBA guys couldn't live with it. They actually want the interface to be directly generated from the code.

Did I talk about bi-directional interfaces? Ah yes, REST can't do bi-directional interfaces either. HTTP can, but not REST.

So Stu, I am sorry, a million times sorry. You'll retire before you can build a decent programming model on top of REST. In the meantime, you are creating a spaghetti plate at the scale of the Web. The RESTafarians, have simply have no idea about what it means to build an information system, and I am sorry, but I think you belong to that category. You have given no evidence of a basic programming model and please let's not even talk about CRUD.

So I am sorry, REST as a programming model, REST as middleware, the (other) REST as I call it (to make it clear nothing that I say here applies to Roy's REST), that REST is fraud, a massive fraud, imposing billions and billions of lost productivity in IT, setting us back 10-15 years, and I wish, Stu, that you would either tell people to hold on, until you figure it out, or if you already came to the same conclusion that you would be open to speak about it.

I have written a new post on "Formalizing the API Style" which expands on the ideas presented here.