Yaron Goland has an entertaining post entitled Interoperability Wars - Episode 6 - Part 1 - Revenge of Babble about some of the philosophical discussions we’ve been having at work about the Atom Publishing Protocol (RFC 5023). The entire post is hilarious if you are an XML protocol geek and I recommend reading it. The following excerpt is a good starting point for another discussion about APP’s suitability as a general purpose editing protocol for the Web. Yaron writes

Emperor Babble: Excellent, Weasdel's death will serve us well in lulling the forces of interoperability into thinking they are making progress. Welcome Restafarian, it is time you understood your true place in my plans.

Luke Restafarian: Do what you want to me emperor, but the noble cause of interoperability will prevail!

The Emperor turns to the center of the chamber where a form, almost blinding in its clarity, takes shape:

GET /someuser/profile HTTP/1.1
host: example.com
content-type: application/xml

<profile xmlns="http://example.com">
<professional>
<workTitle>…</workTitle>

</professional>
<personal>
<spouseName>

</spouseName>

</personal>
<clothingPreferences>
<favoriteColors>
<shirts>…</shirts>

</favoriteColors>

</clothingPreferences>

</profile>

Darth Sudsy is momentarily taken aback from the appearance of the pure interoperable data while Luke Restafarian seems strengthened by it. The Emperor turns back to Luke and then speaks.

Emperor Babble: I see it strengthens you Restafarian, no matter. But look again at your blessed interoperability.

When the Emperor turns back to the form the form begins to morph, growing darker and more sinister:

GET /someuser/profileFeed HTTP/1.1
host: example.com
content-type: application/ATOM+xml

<feed xmlns="http://www.w3.org/2005/Atom">
<title/>
<updated>2000-01-01T00:00:00Z</updated>
<author>
<name/>
</author>
<id>http://www.example.com/someuser/profileFeed</id>
<category scheme="http://example.com/categories" term="Profile"/>
<entry>
<title/>
<id>http://www.example.com/someuser/profilesFeed/professional</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<workTitle>…</workTitle>

</content>
</entry>
<entry>
<title/>
<id>http://www.example.com/someuser/profilesFeed/personal</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<spouseName>

</spouseName>

</content>
</entry>
<entry>
<title/>
<id>http://www.example.com/someuser/profilesFeed/clothingPreferences</id>
<updated>2000-01-01T00:00:00Z</updated>
<content type="Application/XML" xmlns:e="http://example.com">
<favoriteColors>
<shirts>…</shirts>

</favoriteColors>

</content>
</entry>
</feed>

Luke, having recognized the syntax, clearly expects to get another wave of strength but instead he feels sickly. The emperor, looking slightly stronger, turns from the form to look at Luke.

Luke Restafarian: What have you done? That structure is clearly taken from Princess Ape-Pea's system, she is a true follower of interoperability, I should be getting stronger but somehow it's making me ill.

Emperor Babble: You begin to understand young Restafarian. Used properly Princess Ape-Pea's system does indeed honor all the qualities of rich interoperability. But look more carefully at this particular example of her system. Is it not beautiful? Its needless complexity, its useless elements, its bloated form, they herald true incompatibility. No developer will be able to manually deal with such an incomprehensible monstrosity. We have taken your pure interoperable data and hidden it in a mud of useless scaffolding. Princess Ape-Pea and your other minions will accept this poison as adhering to your precious principles of interoperability but in point of fact by turning Princess Ape-Pea's system into a generic structured data manipulation protocol we have forced the data to contort into unnatural forms that are so hard to read, so difficult to understand that no programmer will ever deal with it directly. We will see no repeat of those damned Hit-Tip Knights building their own stacks in a matter of hours in order to enable basic interoperability. In this new world even the most trivial data visualizations and manipulations become a nightmare. Instead of simple transversals of hierarchical structures programmers will be forced into a morass of categories and artificial entry containers. Behold!

Yaron’s point is that since Atom is primarily designed for representing streams of microcontent, the only way to represent other types of data in Atom is to tunnel them as custom XML formats or proprietary extensions to Atom. At this point you’ve added an unnecessary layer of complexity.

The same thing applies to the actual Atom Publishing Protocol. The current design requires clients to use optimistic concurrency to handle conflicts on updates which seems like unnecessary complexity to push to clients as opposed to a “last update wins” scheme. Unfortunately, APP’s interaction model doesn’t support granular updates which means such a scheme isn’t supported by the current design. A number of APP experts have realized this deficiency as you can see from James Snell of IBM’s post entitled Beyond APP - Partial updates and John Panzer of Google’s post entitled RESTful partial updates: PATCH+Ranges.

A potential counter argument that can be presented when pointing these deficiencies of the Atom Publishing Protocol when used outside it’s core scenarios of microcontent publishing is that Google exposes all their services using GData which is effectively APP. This is true, but there are a couple of points to consider

  1. Google had to embrace and extend the Atom format with several proprietary extensions to represent data that was not simply microcontent.
  2. APP experts do not recommend embracing and extending the Atom format the way Google has done since it obviously fragments interoperability. See Joe Gregorio’s post entitled In which we narrowly save Dare from inventing his own publishing protocol and James Snell’s post entitled Silly for more details. 
  3. Such practices leads to a world where we have applications that can only speak Google’s flavor of Atom. I remember when it used to be a bad thing when Microsoft did this but for some reason Google gets a pass.
  4. Google has recognized the additional complexity they’ve created and now ship GData client libraries for several popular platforms which they recommend over talking directly to the protocol. I don’t know about you but I don’t need a vendor specific client library to process RSS feeds or access WebDAV resources, so why do I need one to talk to Google’s APP implementation?

It seems that while we weren’t looking, Google move us a step away from a world of simple, protocol-based interoperability on the Web to one based on running the right platform with the right libraries

Usually I wouldn’t care about whatever bad decisions the folks at Google are making with their API platform. However the problem is that it sends out the wrong message to other Web companies that are building Web APIs. The message that it’s all about embracing and extending Internet standards with interoperability being based on everyone running sanctioned client libraries instead of via simple, RESTful protocols is harmful to the Internet. Unfortunately, this harkens to the bad old days of Microsoft and I’d hate for us to begin a race to the bottom in this arena.

On the other hand, arguing about XML data formats and RESTful protocols are all variations of arguments about what color to paint the bike shed. At the end of the day, the important things are (i) building a compelling end user service and (ii) exposing compelling functionality via APIs to that service. The Facebook REST APIs are a clusterfuck of inconsistency while the Flickr APIs are impressive in how they push irrelevant details of the internals of the service into the developer API (NSIDs anyone?). However both of these APIs are massively popular.

From that perspective, what Google has done with GData is smart in that by standardizing on it even though it isn’t the right tool for the job, they’ve skipped the sorts of ridiculous what-color-to-paint-the-bike-shed discussions that prompted Yaron to write his blog post in the first place. Wink

With that out of the way they can focus on building compelling Web services and exposing interesting functionality via their APIs. By the way, am I the last person to find out about the YouTube GData API?.

Now playing: DJ Khaled - I'm So Hood (feat. T-Pain, Trick Daddy, Rick Ross & Plies)


 

Wednesday, October 10, 2007 9:18:27 AM (GMT Daylight Time, UTC+01:00)
"...requires clients to use optimistic concurrency..."

This is incorrect. RFC5023 (AtomPub) doesn't make any requirements above what RFC2616 (HTTP/1.1) defines. Thus, if you want to implement "last update wins", you're free to do so. Actually, I would expect that to be the common way how implementations work.

Best regards, Julian
Julian Reschke
Wednesday, October 10, 2007 1:08:41 PM (GMT Daylight Time, UTC+01:00)
Julian,
I agree that most APP implementations will just use "last update wins". However if a service cares about the lost update problem [which large scale services with lots of diverse clients will] then the sanctioned way of doing this with APP is via optimistic concurrency.
Wednesday, October 10, 2007 4:27:06 PM (GMT Daylight Time, UTC+01:00)
Dare,

I'm not sure what your point is. AtomPub is just HTTP, so you can have pessimistic locking (LOCK), optimistic locking (If-Match:), or no locking at all. You also can have full update (PUT) or partial updates (PATCH).

Yes, I know that PATCH isn't finished yet, but better a generic solution that can be applied to *any* HTTP resource than something specific to the protocol.

Best regards, Julian
Julian Reschke
Wednesday, October 10, 2007 6:22:58 PM (GMT Daylight Time, UTC+01:00)
I'm curious as to why you didn't throw in PROPPATCH in that list. After all, isn't WebDAV just HTTP as well?

Anyway, the APP defines a particular set of interactions within the constraints of HTTP. You are admitting that none of the currently supported interactions (LOCK, PUT, If-Match) meets the problem case I outlined above where you have diverse, disconnected clients and care about the lost update problem.

If-Match is problematic because it means you have to trust clients that you have no control over to implement the correct merge semantics. PATCH would work well, except there is no spec for PATCH so it seems facetious to throw it in as an option.
Wednesday, October 10, 2007 10:44:30 PM (GMT Daylight Time, UTC+01:00)
I'm not sure how PROPPATCH would be relevant here, unless you want to model the data as WebDAV properties rather than content.

Anyway, it seems your focus was on the fact that AtomPub isn't well suited for hierarchical data, with which I happen to agree.

I just was confused that you were making that statement about required optimistic locking in this context.

Best regards, Julian

Julian Reschke
Wednesday, October 10, 2007 11:33:30 PM (GMT Daylight Time, UTC+01:00)
FWIW, there is a work-in-progress I-D for PATCH. Stress on the "work-in-progress"
Monday, October 15, 2007 9:12:57 AM (GMT Daylight Time, UTC+01:00)
Well, it's just DOM-deconstructable XML text.
- Using PHP DOM classes, it's quite easy (though time-consuming) to sculpt one proprietary protocol into any text shape - even those with whitespace sensitivity such as OOXML.
- It's only when you start using proprietary compression schemes and DRM that interoperability really starts to suffer.
ZowHyre
Comments are closed.