My recent post, Explaining REST To Damien Katz, led to some insightful comments from Dave Winer and Tim Bray about what proponents of building RESTful Web services can learn from remote procedure call (RPC) technologies like SOAP and XML-RPC. 

In his post, Dare left something out (and it's important) Dave Winer wrote

Really ought to include it in your thinking, Dare and everyone else. You're missing out on something that works really well. You should at least learn the lessons and add to REST what it needs to catch up with XML-RPC. Seriously.

What's missing in REST, btw, is a standard method of serializing structs, lists and scalar types. The languages we use have a lot more in common than you might think. We're all writing code, again and again, every time we support a new interface that could be written once and then baked into the kernels of our languages, and then our operating systems. Apple actually did this with Mac OS, XML-RPC support is baked in. So did Python. So if you think it's just me saying this, you should take another look.

Dave has a valid point, a lot of the time communication between distributed systems is simply passing around serialized objects and/or collections of objects. In such cases, having a lightweight standard representation for serialized objects which is supported on multiple platforms would be beneficial. Where Dave goes astray is in his assertion that no such technology exists for developers building RESTful Web services. Actually one does, and it is widely deployed on the Web today. JavaScript Object Notation (JSON) which is described in RFC 4627 is a widely deployed and well-defined media type (application/json) for representing serialized structs, lists and scalar values on the Web. 

There are libraries for processing JSON on practically every popular platform from "corporate" platforms like Java and the .NET Framework to Web scripting languages like Python and Ruby. In addition, JSON is attractive because it is natively available in modern Web browsers that support JavaScript which means you can use it to build services that can be consumed by Web browsers, other Web services or desktop applications with a single end point.

Tim Bray cautioned REST proponents against blindly rejecting the features typically associated with RPC systems and SOAP/WS-* in his post REST Questions where he wrote

Has REST Been Fortunate in its Enemies? · I have been among the most vocal of those sneering at WS-*, and I’m comfortable with what I’ve said. But that doesn’t prove that the core WS-* ideas are wrong. Here are some of the handicaps WS-* struggled under:

  • Lousy foundational technologies (XSD and WSDL).

  • A Microsoft/IBM-driven process that was cripplingly product-linked and political.

  • Designers undereducated in the realities of the Web as it is.

  • Unnecessary detours into Architecture Astronautics.

As a result, we should be really careful about drawing lessons from the failure of WS-*. Specifically:

  • Just because the XSD type system is malformed, you can’t conclude that the notion of schema-driven mapping between program data types and representation payloads is harmful.

  • Just because WSDL is a crock, you can’t conclude that exposing a machine-readable contract for a service is a necessarily bad idea.

  • Just because UDDI never took off, you can’t conclude that service registries are dumb.

  • Just because SOAP has a damaging MustUnderstand facility and grew a lot of ancillary specification hair, you can’t conclude that some sort of re-usable payload wrapper is necessarily a dead-end path.

  • Just because the WS-* security specifications are overengineered and based on a shaky canonicalization spec, you can’t conclude that message-level security and signing aren’t sometimes real important.

And so on. I personally tend to think that schema-driven mapping is hopeless, contracts are interesting, registries are a fantasy, and payload wrappers are very promising. But I don’t think that the history of WS-* is a very good argument for any of those positions.

In a lot of situations where applications consume XML, the first thing the application does is convert the XML into an object graph representative of the business logic of the application. The SOAP/WS-* way of doing this was to define an XSD schema for the XML payload and then use some object<->XML mapping layer to convert the XML to objects. The problem with this approach was that there is a fundamental impedance mismatch between XSD types and OO types which led to horrible interoperability problems since no two platforms could agree on how to map the various esoteric type system features of XSD into the structs, lists and scalar types that are the building block of all OO type systems. However these problems go away if you use a data format that was explicitly designed for describing serialized data objects like JSON.

Providing a machine readable description of a service's end points is useful, especially on the Web where multiple services may expose the same interface. For example, when you visit my weblog at http://www.25hoursaday.com/weblog/ using Firefox 2 or higher and Internet Explorer 7 or higher the browser immediately lights up with a feed icon which allows you to subscribe to my Atom feed from your Web browser. This happens because I've provided a machine readable description of my feed end points on my blog. The Atom Publishing Protocol, which is one of the most well-designed RESTful protocols out there, has a notion of service documents which enable client applications to discover the capabilities and locations of the various end points of the service.

If you put together the notion of service documents with using JSON as the payload format for a service endpoint, you're close to getting the touted programmer friendliness of RPC technologies like XML-RPC & SOAP/WSDL while still building a RESTful service which works with the Web instead of against it.

The only problem is how to deal with statically typed languages like C# and Java. These languages need the types of the objects that application will consume from a Web service defined up front. So if we could just figure out how to come up with service documents for JSON services that included the notion of a class definition, we could pretty much have our cake and eat it to with regards to getting the ease of use of an RPC system while building a RESTful service.

If this sounds interesting to you, then you should go over and read Joe Gregorio's original post on RESTful JSON and then join the restful-json mailing list. Joe's proposal is on the right path although I think he is letting his background as an editor of the Atom Publishing Protocol specification skew his thinking with regards to what developers would find most useful from a Json Publishing Protocol (JsonPub).

Now Playing: G-Unit - Beg For Mercy


 

Sunday, August 24, 2008 5:16:34 PM (GMT Daylight Time, UTC+01:00)
JSON is still only about 70% of the way there. XML is a node-labeled tree, while JSON is an edge-labeled tree. Unfortunately, what we want all too often an edge-labeled graph.

At least it doesn't include of most of the odd XSD datatypes, and doesn't have the namespace roundtripping issues that XML has (though they are useful when you're looking at XML as a document).
Sunday, August 24, 2008 5:46:49 PM (GMT Daylight Time, UTC+01:00)
"Unfortunately, what we want all too often an edge-labeled graph."

Barry, that's RDF, if you're willing to use it. Afaict, developers find that even harder to work with than XML.
Sunday, August 24, 2008 8:28:22 PM (GMT Daylight Time, UTC+01:00)
RDF is simple. It's the dropping their XML hierarchy mentality is what people have such a hard time with. Always remember, it's a graph. There are a few notations for RDF, XML just happens to be the one people use the most. N3 is the most readable, while Turtle is very terse and resembles JSON (except it's not a hack).

Machine-readable contract definitions are a *good* thing. A lot of smart, brilliant people have put a lot of cerebral CPU cycles into WSDL and XSD. There's no need to start over. I've said it a few times on your blog Dare, and I'll say it again; a machine-readable contract that allows a *computer* (NOT human) to quantify the 1) types of operations, 2) available operations, and whether or not an endpoint 3) supports a particular operation is a MUST in a "web services stack".

Tim bray makes some pretty bold assertions, like "Lousy foundational technologies (XSD and WSDL)." but he doesn't quantify why they're lousy. I think that's a cop-out.

RESTful services are easy to understand, but they lack some fundamental constructs.

For example; Let's say I have a REST endpoint in the form of http://www.25hoursaday.com/api/CalculateSomeValue/. The contract for CalculateSomeValue needs two parameters; a ComplexObject and an int32. And it returns a number (some kind of calculation off of the two params). So for the C#/Java folks it would look like this: int CalculateSomeValue(ComplexObject obj, int Number);

Using RESTful methods, as far as I can tell there's no clean/efficient way to transfer state (pass the args) to the method without some kind of *method specific* construct. If the method *only* took a ComplexObject, then you could serialize it using whatever notation you want and just POST it. Or if it *just* took an integer, you could do pass it using application/x-www-form-urlencoded and just URL encode the POST. But there's no good way (as far as I can tell, at least) to pass both args without some kind of method specific construct--which totally sucks. FWIW, WS-* suffers from the same limitation, which is why in the Visual Studio auto-generated WSDLs you'll see things like CalculateSomeValueRequest and CalculateSomeValueResponse.

For what it's worth, I think the Semantic Web and it's associated constructs (RDF, SPARQL, OWL) solve a lot of the problems you're currently wrestling with.

My $0.03.
Josh Jonte
Sunday, August 24, 2008 9:05:24 PM (GMT Daylight Time, UTC+01:00)
So if we could just figure out how to come up with service documents for JSON services that included the notion of a class definition, we could pretty much have our cake and eat it to with regards to getting the ease of use of an RPC system while building a RESTful service.


Once you've done that, you'd be about 60% of the way to getting what Protocol Buffers already have. Add a extremely efficient binary format that is isomorphic to the text format and you'll be 75% there (the binary format, by the way, is not the only serialization format Protocol Buffers have -- they have a lossless text format that is much like JSON). Once you've created a compiler for your message description language that can emit code for static languages, you'll be 90% of the way there.

This is why it is so befuddling why you decided you don't like protocol buffers. Don't you realize that what you are proposing is the same thing? Maybe you didn't realize that protocol buffers have a text format.

I like JSON well enough. But if you compare JSON and Protocol Buffers, JSON's about 5 steps behind. JSON's only advantage is that it's the incumbant.
Monday, August 25, 2008 2:58:37 AM (GMT Daylight Time, UTC+01:00)
Dare,
Have you gotten a chance to look at some of the incredible work Kris Zyp has done on jsON? He created his own jsON REST proposal with a complete working framework YEARS ago. He has proposed a jsON schema, query, path, referencing, and json in COMET servers.
Check him out on http://json.Com
(Disclaimer: I own the domain, but Kris uses it to promote json for Free)
Monday, August 25, 2008 6:02:15 AM (GMT Daylight Time, UTC+01:00)
Somewhat lost in this discussion is that Damien did implement a restful JSON API in couchdb:
"For CouchDB, REST makes absolutely insanely perfect sense"
It implements Get, POST, PUT and DELTE and uses JSON. CouchDB documents ARE JSON so the transport and persistence data model are the same:
An introduction to the CouchDB HTTP document API
Monday, August 25, 2008 11:32:05 AM (GMT Daylight Time, UTC+01:00)
The only problem is how to deal with statically typed languages like C# and Java. These languages need the types of the objects that application will consume from a Web service defined up front. So if we could just figure out how to come up with service documents for JSON services that included the notion of a class definition [...]


I think the answer is: don't. Rather extend Java and/or C# to allow programmers escape from the type system at places where they need to. Like in "Fan" (the programming language), provide a '->' operator that calls a method that is not statically known, or delegates to "methodMissing" (or similar).

Static typing is really nice and sound in many places in applications, but I think there are pretty convincing arguments that it's not a good thing to deploy in distributed systems.
Thursday, August 28, 2008 9:18:43 PM (GMT Daylight Time, UTC+01:00)
Why can't programmers developing RESTful web services use XML as a somewhat lightweight standard representation for serialized objects which is supported on multiple platforms? JSON has some serious security drawbacks compared to XML. I wouldn't use JSON if there was any personal data involved or business data requiring protection. Please see http://www.xml.com/cs/user/view/cs_msg/3370 or http://directwebremoting.org/blog/joe/2007/03/05/json_is_not_as_safe_as_people_think_it_is.html for two good explanations of the security problems you can have if you select to use JSON instead of XML.
James Orenchak
Comments are closed.