August 17, 2008
@ 12:33 PM

Damien Katz recently caused a stir on a bunch of the blogs I read with his post entitled REST, I just don't get it where he wrote

As the guy who created CouchDB, I should be a big cheerleader for RESTful architectures. But the truth is, I just don't get it.

For CouchDB, REST makes absolutely insanely perfect sense. Read a document, edit, put the document back. Beautiful. But for most applications, enterprise or not, I don't see what the big win is.

I know what is wrong with SOAP, and it has everything to do with unnecessary complexity and solving the same problems twice. But what is the big advantage of making all your calls into GET PUT and DELETE? If POST can handle everything you need, then what's the problem?

I guess what I mean to say is just because SOAP is a disaster, doesn't somehow make REST the answer. Simpler is better, and REST is generally simpler than SOAP. But there is nothing wrong with a plain old POST as an RPC call. If its easy to make all your calls conform to the RESTful verb architecture, then that's good, I guess.

His post made the rounds on the expected social news sites like programming.reddit and Hacker News, where I was amused to note that my blog is now being used as an example of silly REST dogma by REST skeptics in such discussions. From reading the Damien's post and the various comments in response, it seems clear that there are several misconceptions as to what constitutes REST and what its benefits are from a practical perspective.

Background: The Origins of REST vs. SOAP

The Representational State Transfer (REST) architectural style was first described in Chapter 5 of Roy Fielding's Ph.D dissertation published in 2000. It describes the architecture of the Web from the perspective of one of the authors of the HTTP 1.1 specification which was published the year before in 1999. Around the same time Don Box, Dave Winer and a bunch of folks at Microsoft came up with the Simple Object Access Protocol (SOAP) which they intended to be the standard protocol for building distributed applications on the Web.

Over the following years SOAP was embraced by pretty much every major enterprise software vendor and was being pushed hard by the W3C as the way to build distributed applications on the Web. However a lot of these vendors weren't really interested in building software for the Web but instead were more interested in porting all of their technologies and scenarios from enterprise integration technologies like CORBA to using buzzword compliant XML. This led to a bunch of additional specifications like XSD (type system), WSDL (IDL) and UDDI (naming/trading service). Developers initially embraced these technologies enthusiastically which led to the enterprise software vendors pumping out dozens of WS-* specifications. During this period not many thought or talked much about REST since no one talks about boring Ph.D dissertations. 

In 2002, a canary in the coal mine emerged in the form of Mark Baker. On mailing lists frequented by Web services types such as xml-dev and xml-dist-app, Mark began to persistently point out that SOAP was built on a bad foundation because it fundamentally ignored the architecture of the Web as defined by Roy Fielding's thesis. At first a lot of people labeled mark as a kook or malcontent for questioning the trend of the moment.

By 2005, the industry had enough  experience with SOAP to start seeing real problems using at as a way to build distributed applications on the Web. By that year many developers had started hearing stories like Nelson Minar's presentation on the problems Google had seen with SOAP based Web services and sought a simpler alternative. This is when the seeds of Mark Baker's evangelism of Roy's dissertation eventually bore fruit with the Web developer community.

However a Ph.D dissertation is hard to digest. So the message of REST started getting repackaged into simpler, bite-sized chunks but the meat of the message started getting lost along the way. Which led to several misconceptions about what REST actually is being propagated across the Web developer community.

Misconceptions About the REST Architectural Style

With that out of the way I can address the straw man argument presented in Damien's post. Damien states that building a RESTful Web Service is about using the HTTP PUT and DELETE methods instead of using HTTP POST when designing a Web API. In fact, he goes further to describe it as "the RESTful verb architecture" implying that choice of HTTP verbs that your service supports is the essence of REST.

This is incorrect.

Q: What is the Essence of REST? A: The Uniform Interface

REST explains how the Web works by defining the set of constraints on the various components in the current Web architecture. These constraints include

  • interaction is based on the client-server architectural style. User agents like Web browsers, RSS readers, Twitter clients, etc are examples of Web clients which talk to various Web servers without having a tight coupling to the internal implementation of the server.

  • communication between the client and server is stateless. The benefits of HTTP being a primarily stateless protocol is that statelessness increases scalability and reliability of services at the cost of introducing some complexity on the part of the client.

  • the Web architecture supports caching by requiring that requests that are cacheable or non-cacheable are labeled as such (i.e. via HTTP method and various caching related headers).

  • there is a uniform interface between components which allows them to communicate in a standard way but may not be optimized for specific application scenarios. There are four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.

  • there can be multiple layers between client and server which act as intermediaries (e.g. proxies, gateways, load balancers, etc) without this being obvious to the requesting client or accepting server.

When you read the above list, the first thing you will note is that it describes the architecture of the World Wide Web. It doesn't describe the architecture of "a typical enterprise" or the internals of a cloud computing application.

Building a RESTful Web Service simply means paying attention to these characteristics of the Web. As you read them, some practical guidelines start becoming obvious. For example, if you want to take advantage of all the caching infrastructure that is built into the Web infrastructure, then you should use HTTP GET for services that retrieve data. This is just one of the many things Web Services built on SOAP got wrong.

The uniform interface constraints describe how a service built for the Web can be a good participant in the Web architecture. These constraints are described briefly as follows

  1. Identification of resources: A resource is any information item that can be named and represented (e.g. a document, a stock price at a given point in time, the current weather in Las Vegas, etc). Resources in your service should be identified using URIs.

  2. Manipulation of resources via representations: A representation is the physical representation of a resource and should correspond to a valid media type. Using standard media types as the data formats behind your service increases the reach of your service by making it accessible to a wide range of potential clients. Interaction with the resource should be based on retrieval and manipulation of the representation of the resource identified by its URI.

  3. Self-descriptive messages: Following the principles of statelessness in your service's interactions, using standard media types and correctly indicating the cacheability of messages via HTTP method usage and control headers ensures that messages are self descriptive. Self descriptive messages make it possible for messages to be processed by intermediaries between the client and server without impacting either.

  4. Hypermedia as the engine of application state: Application state should be expressed using URIs and hyperlinks to transition between states. This is probably the most controversial and least understood of the architectural constraints set forth in Roy Fielding's dissertation. In fact, Fielding's dissertation contains an explicit arguments against using HTTP cookies for representing application state to hammer this point home yet it is often ignored.

Benefits of Conforming to REST and the Uniform Interface to Web Developers

At this point, the benefits of building RESTful services for the Web should be self evident. The Web has a particular architecture and it makes sense that if you are deploying a service or API on the Web then it should take advantage of this architecture instead of fighting against it. There are millions of deployed clients, servers and intermediaries that support REST and it makes sense to be compatible with their expectations.

This doesn't mean you have to use DELETE and PUT when POST might suffice. It does mean understanding the difference between using POST versus using PUT to other participants in the Web architecture. Specifically, that PUT is idempotent while POST is not so a client of your service can assume that performing the same PUT two or three times in a row has the same effect as doing it once but cannot assume that for POST. Of course, it is up to you as a Web service developer to decide if you want your service to provide a more explicit contract with clients or not. What is important to note is that there is a practical reason for making the distinction between which HTTP verbs you should support.

There are other practical things to be mindful of as well to ensure that your service is being a good participant in the Web ecosystem. These include using GET instead of POST when retrieving a resource and properly utilizing the caching related headers as needed (If-Modified-Since/Last-Modified, If-None-Match/ETag, Cache-Control),  learning to utilize HTTP status codes correctly (i.e. errors shouldn't return HTTP 200 OK), keeping your design stateless to enable it to scale more cheaply and so on. The increased costs, scalability concerns and complexity that developers face when they ignore these principles is captured in blog posts and articles all over the Web such as Session State is Evil and Cache SOAP services on the client side. You don't have to look hard to find them. What most developers don't realize is that the problems they are facing are because they aren't keeping RESTful principles in mind.

Don't fight the Web, embrace it.

FURTHER READING

Now Playing: Public Enemy - Don't Believe the Hype


 

Sunday, 17 August 2008 23:36:12 (GMT Daylight Time, UTC+01:00)
Especially because of posts like this, I am so glad that you returned to blogging your thoughts Dare.

Clear and concise. Keep it up!
Monday, 18 August 2008 07:44:27 (GMT Daylight Time, UTC+01:00)
The more people try and make a case for rest the sillier it gets. Im on Damiens' side on this one.
REST is a NON-architecture.
bob
Monday, 18 August 2008 09:50:50 (GMT Daylight Time, UTC+01:00)
Thanks for writing this post, clear and very useful.
vds
Monday, 18 August 2008 20:24:49 (GMT Daylight Time, UTC+01:00)
Seems like reasonable arguments in favor of REST to me. Acknowledged in this highlight of the week :-).
Wednesday, 20 August 2008 05:16:04 (GMT Daylight Time, UTC+01:00)
Thanks for the explanation and the links (again). I still have some conceptual problems with the REST movement:

1. There still isn't (as far as I can tell) any good way to describe a REST service's resources and representations. This means (especially in less-flexible languages such as Java or C#) I have to do a lot of work each time I want to use a REST service. Essentially I have to write my own client (unless one already exists) that has all the information about what resources exist at what URIs and how their representations can be serialized into objects and back. WSDL and XSD certainly aren't pretty, but at least in theory I get clients for free. This isn't such a problem in a language like Ruby where it's relatively easy to deal with HTTP calls and XML serialization/deserialization, but especially if I want the representations repackaged as "real" objects, and want to apply caching or retry logic, this gets complicated quickly.
2. REST is great at modeling a set of objects and CRUD operations, but isn't good at representing RPC-style services. Where's the simple non-SOAP system for modeling those types of service? When I've seen those services implemented in a REST style they always come out very contorted. Is the idea that I should represent "operations" as just a POST-only resource?
3. There seems to be a lot of confusion on how to model and break up your resources and representations. Get/Set operations can be messy since you often have a much larger document but you only want to update a small piece of it.

I guess a lot of my problems are tied up with how to express the resources and their representations. The "go with the HTTP flow" part makes perfect sense to me. How to represent everything I want as a collection of URLs, four HTTP verbs, and a mess of XML/JSON/whatever is much harder.
Saturday, 23 August 2008 12:45:04 (GMT Daylight Time, UTC+01:00)
In 2002, a canary in the coal mine emerged in the form of Mark Baker. On mailing lists frequented by Web services types such as xml-dev and xml-dist-app, Mark began to persistently point out that SOAP was built on a bad foundation because it fundamentally ignored the architecture of the Web as defined by Roy Fielding's thesis. At first a lot of people labeled mark as a kook or malcontent for questioning the trend of the moment.


You can't be serious about this, do you? SOAP never intended to follow the architecture of the Web. SOAP is a messaging protocol and its independence of transport protocol was a major design goal. One of the nice things you can do with SOAP is tunneling through HTTP. That way it allows doing messaging without having severe difficulties with firewalls etc. as this was already "solved" for HTTP. So it uses HTTP for its purpose (i.e. as a transport for messaging). Of course this has nothing to do with REST, but hey, its messaging and not ROA. So what is actually the problem. Nobody would argue that JMS is crap, just because its not RESTful. Different solutions for different problems. And REST is definitely not messaging.

When I read your post, then I got the impression, that the whole REST vs. SOAP debate (which is in my opinion the wrong debate, SOA vs ROA would be more valuable) is a misunderstanding based on the word "Web" in "Web Services". As we all know that WS-* is a foundation about interoperable, transport neutral messaging, and not about using HTTP in a RESTful manner. I agree that the "Web" is misleading, you should simply ignore it. That being said, I'm wondering what then remains. I guess not so much. I'm really convinced that there is enough room for both worlds. I'm not really surprised that Google found out that SOAP/WSDL was not the best choice to provide a resource oriented access. I'm also sure that if you have complex heterogeneous environments, you'll be happy to rely on message routing with end-to-end QoS. The debate on which approach is in general the best is awkward, the debate on when to use what is IMO the interesting one.
Tammo van Lessen
Tuesday, 26 August 2008 01:36:50 (GMT Daylight Time, UTC+01:00)
Tammo - something Dare didn't mention was that pointing out that there was no "Web" in "Web services" was only part of my message. The rest of the message was that the Web is a desirable platform to build solutions to problems that Web services were trying to solve.

Oh, and Dare, it was actually 2000 when I officially began SOAP bashing;

http://discuss.develop.com/archives/wa.exe?A2=ind0003&L=soap&T=0&F=&S=&P=35306
Comments are closed.