It is no secret that I am not a big fan of applying Roy’s REST to something other than the Web Architecture. Hipsters have pushed “REST” to architect distributed systems and craft APIs, but after a decade of RESTfulness, the RESTafarians have little to show for: the so-called “uniform interface” is not that uniform (it’s rather verbose), “resource-orientation” is just a fancy name for CRUD, schema-less contracts lead to … Swagger and … back to code generation, and so forth. I can’t think of a single recommendation from Tilkov, Vinoski, Bray, Burke… that turned out to be correct.
Since by definition a hipster would never admit defeat, the only strategy possible is the “fuite en avant“: everything else was wrong, no worries, we got it right this time: the correct way to do REST is “hypermedia”. Right?
I find it fascinating that the hardcore RESTafarians like Stefan Tilkov or Mike Amundsen are now using some of my arguments to “REST” correctly as they introduce the concept of a state machine to identify the verbs, yes, you heard it right, the verbs of “the” once thought to be uniform interface.
In case you are not familiar with the role of state machines in computation, please I beg you to read this outstanding paper from Dr. Lamport, on Computation and State Machines. Anyone who writes code and designs interfaces must read it.
With that in mind, let’s explore the use of Hypermedia in distributed systems (M2M). Five years ago, I published this post detailing the metamodel behind REST to surface its semantics (interestingly RESTafarians are allergic to metamodels…):
There are three parts to REST:
- REST core (derived from Roy’s Thesis),
- What I call the practical REST which added query parameters, collections and properties of resources.
- What I call the “full REST” which of course includes states, transitions and actions.
If you disagree with me on the “full REST” and the connection of non-deterministic state machines with hypermedia, look no further than the great work of Ivan Zuzak et al on the “Formal Modeling of RESTful Systems using State Machines”.
I do agree that Hypermedia is the keystone of the architecture of the Web and without it, the Web, or what’s left of it, would look very different, but unless you never wrote a single line of code in your life, you’d quickly spot that Hypermedia works because there is a user in the loop, in other words, a human, which is capable of interpreting the links and deciding which action to take next. Arbitrary strings of characters are as good as graffiti on the wall facing a computer’s “web”cam when it comes to helping a machine decide what to do next.
What can Hypermedia teach us about API design? Hypermedia clearly shows that the interface to a resource is not “uniform” never has been, never will be. Duh!! At last, after nearly a decade of “nounsense”, Stefan and Mike use verbs… how refreshing.
Is hypermedia hyped today with the same kind of bullshit intensity we heard when REST started? you bet! Look no further than this summary of RESTFest. Imagine… with Hypermedia you can build “bots”, which, without the knowledge of chess, can play chess against each other… OMFG, Hypermedia can even do machine learning. I can’t wait to see how Amazon’s drones and Google’s self-driving car use Hypermedia.
But, to be fair, when you take away the people who are financially and/or emotionally attached to REST, you get a reasonable set of arguments behind the use of hypermedia, or so it looks. For instance, Glenn Block wrote this post in 2012, where he explains:
In a hypermedia based system, links are generated based on a state machine. Depending on the state of the resource, different links apply. [This means that] you will have to deal with how to handle the generation [and interpretation] of links.
Model-driven engineering has been around for quite some time, and I have been building model-driven software since 1991 (thanks to NeXTStep and Interface Builder, and things like EOF…). Embedding metadata in a server’s response for the client to interpret has probably been around since the 60s, I would guess, the first distributed system ever built was using that “pattern” already.
So what do you gain by paying the penalty of inflating your responses with next-action metadata? Can you spontaneously teach a machine how to play chess that way? Well, no… From Roy’s thesis:
Machines can follow links when they understand the data format and relationship types.
Yes, a small detail that a lot of hipters fail to surface is that, machines need to form a shared understanding (out-of-band) of the data formats and the relationship types. Since they need to form a shared understanding in the first place, what could be the value of redundently Hypermediating machine responses in M2M scenarios?
First, in complex non-deterministic state machines, it might be great to give some hints (a.k.a. constraints) to the client as to what is possible in case the developer(s) of the client(s) can’t read the doc and create a client that works properly. Ah, but I almost forgot, you remember what the Vinoskis of the world were telling us in 2007? that client SDKs were no longer needed with REST, that simple human readable doc was enough? How did that pan out? Yep, the serious API providers have to hand-code at great cost what WSDL was generating for free. So it is unlikely that this kind of metadata will ever become a major driver for Hypermedia adoption, because the hypermedia will be buried behind the client SDK, and since the API providers write both the SDK and the APIs, they don’t really need hypermedia to support the shared understanding between client and API.
Second, one could use the link metadata to construct UIs (with queries, actions…). RoFL!! You mean the RESTafarians could convince someone to rewrite a proprietary HTML engine, just for the privilege of using hypermedia? I am sure some people will pay top $$$ just for that privilege, just ask some of the people who paid north of $300/hr to learn how to convert verbs into nouns. After all they have already convinced our industry that it was cool to hand-code ESBs, client SDKs… So why not add an HTML engine, if they can find people who are ready to pay for learning how to do that?
The problem in finding value for M2M-Hypermedia, however, is not just shared understanding, it is shared state. Again, most of the Web’s architecture was built with the user in mind, when we talk about the “application” state, we talk about the user session, not much more. When we talk about M2M, we talk about many machines interacting with (and therefore changing the state of) each other (imagine a server booting, what does it mean to hold a representation of its state?). REST is essentially client/server, and therefore not an architecture suitable for distributed computing. REST’s fundamental assumption is that the state you are handed off changes infrequently (as in never) and even if it changes, the stale state is still actionable (as in you can navigate to something that makes sense regardless). Web Apps have been architected with the constraint that the state is solely changed under the circumstances of the client.
In M2M, state is likely to be stale, irrelevant, because another machine would have changed that state before any action could be taken on a given representation. IoT is not the WoT, otherwise it would be called the Web of Things. So, what is the point of having the knowledge of the “affordances” on a piece of state that is no longer relevant? Interesting question, isn’t it?
So why are the RESTafarians pushing hypermedia so hard today when there is absolutely no value to show for?
The true reason is because REST has a flaw at its core, a major flaw, that the hipsters don’t know how to deal with, perhaps other than by pulling in the machinery of hypermedia. They won’t tell it to you that way, but I reckon most of the hypermedia you will see will be used to deal with that issue.
The problem with REST is that it couples access with identity.
When you write:
you are coupling the identity of the order (123) with the way you access its “state”. That kind of URI can’t be opaque to a machine/client (again all this works nicely when a user is in the loop). So at a minimum, REST needs an identity relationship that provides a iURId (as in: immutable Uniform Resource Identity) that a client can rely on to retrieve that resource at a later time (generally via a mapping from their own internal identity of the resource).
When you don’t have a iURId to rely on, when a new version comes out and all the plain old URIs of the orders you hold on the client side become obsolete.
So, how do you know that these two URIs https://api.blackforest.com/v1/orders/123 and https://api.blackforest.com/v2/orders/123 point to the same order?
This is also true of “views” of a resource, how do you know that these views point to the same order?
Well, you really don’t know they are the same, unless you have a shared understanding with the resource owner that 123 is the identity of the order and https://api.blackforest.com/v1/orders/ is (one of) the way you access it.
Yep, you don’t hear much the hipsters telling you about these little rough edges of REST. My good friend Pete Williams does explain it, but in a different way:
A URI that is constructed by a client constitutes a permanent, potentially huge, commitment by the server. Any resource that may be addressed by the constructed URIs must forever live on that particular server (or set of servers) and the URI patterns must be supported forever
Effectively, you are trading a small one time development cost on the client side for an ongoing, and ever increasing, maintenance cost on the server side.
But who, who in their right mind, would let his client trust and follow a dynamic link based on a label (relationship name)? Are client-side URIs really such an acute problem? Isn’t compatible versioning enough to deal with these server-side cost?
The bottom line is that Hypermedia has zero value, especially in the context of M2M (again, I am not talking about the Web’s Architecture when a human is in the loop). I am certain that someone somewhere will find an application (or two) that shows some value, just like any interesting software pattern, beyond that, hypermedia will be a big waste of time for most people, just like the uniform interface, the coupling of identity with access, http caching, verbs vs nouns, human readable documentation… have been thus far.
Can we just turn the page (pun intended) on REST, once and for all?