I’ve been reading recently that a number of social networking sites are rushing to launch [or re-launch] a widgets platform given the success of the Facebook platform. There have been announcements about a MySpace platform which claim that

it will essentially be a set of APIs and a new markup language that will allow third party developers to create applications that run within MySpace. Developers will be able to include Flash applets, iFrame elements and Javascript snippets in their applications, and access most of the core MySpace resources (profile information, friend list, activity history, etc.). Unlike existing widgets on MySpace, developers will be able to access deep profile and other information about users and bake it into the applications.

Advertising can be included on the application pages (called control pages) and developers will keep 100% of the revenue. Ads may not be placed within widgets that appear on MySpace pages, however.

There have been similar announcements from LinkedIn and Google. The problem is that every one of these widget platforms being proposed by the various social networking sites are incompatible. This means that Web developers has to build a separate application for each of these sites using proprietary technologies (e.g. FBML) and proprietary APIs (e.g. FQL). Since very few Web developers or Web companies will be able to support significant applications on every one of these platforms, the question then becomes “Which platform should you bet on?” and “How do you make the decision to bet on a platform.”

Right now the gold standard in widget platforms for social networking sites is the Facebook platform. There are several reasons for this and competitors planning to build similar platforms need to meet the following criteria.

1. Monetization: Facebook encourages developers to monetize their widgets by placing ads in their widgets. Although Facebook has not actively helped developers by providing an ad platform, there is now a healthy marketplace of Facebook ad networks that developers can choose from. It has even been rumored that Google will be getting in the Facebook ad provider game. +1 to Facebook.

2. Distribution and Reach: A big problem you’ll face when you’ve built a great product is that it is a lot harder than you expect for people to actually find out about and try your product. This means any avenue that increases the potential reach and distribution of your product is bringing money in your pocket. Not only does Facebook have several million active and engaged users, the Facebook platform also provides several mechanisms that encourage the viral spread of applications which developers consistently rave about. No other social networking site’s widget platform even comes close. +1 to Facebook.

3. Access to User Data: Social networking sites are all about connecting people to the people they care about and their interests. This means that applications built on these platforms should be able to determine a user’s friends and interests to be able to give an optimal experience. The Facebook platform is unprecedented in the arena of widget platforms when it comes to the amount of user information it exposes to applications with methods like friends.get, users.getInfo, photos.get and even marketplace.getListings. +1 to Facebook.

4. Ability to Build an Integrated and Immersive Experience: One place where the Facebook platform really changed the game is that widgets weren’t just relegated to a tiny box on the user’s profile like they are on other social networking sites but instead developers could build full blown applications that integrated fully into the Facebook experience. It’s a lot easier to keep users engaged and build non-intrusive advertising into your application if your entire application doesn’t have to fit in some 4” X 4” box on the user’s profile. +1 to Facebook.

5. Applications Shielded from the “Winner’s Curse” of Web Development: The more successful your application becomes on the Web, the more money you have to spend on server related resources. Everyone knows the story of iLike scrambling to borrow money and servers because their Facebook application was more successful than they anticipated. Since a lot of widget developers are not richly funded startups or big companies, they may not be able to bear the costs of actually building a successful Web application without help from the platform vendor. A number of platform vendors provides hosting for static files and data storage APIs although none go as far as full blown application hosting...yet.  +0.5 to Facebook.

From my perspective, if a social networking site’s widget platform doesn’t meet all criteria, then it can’t be considered a real competitor to Facebook’s platform. And as a developer if I had to choose between that platform and Facebook’s, there would be no choice to make.

Now if you can afford multiple development efforts building widgets/applications for several disparate social networking site platforms, the list above is a good starting point for prioritizing the which social networking site’s to build widgets/applications for first.   

Now playing: T.I. - U Don't Know Me

It looks like my favorite team in Windows Live scored another hit with their recent release. You can see it for yourself by going to http://maps.live.com. The product team has a huge list of the new changes in their post Live Search Maps v2 is out! Gemini Launches which list a number of significant features including

• generic directions to a specific location (e.g. if coming from I-5 North vs. if coming from I-5 South) without a specific starting point. I believe this is formally called 1–click directions.
• the ability to have the directions route around bad traffic
• the ability to import GeoRSS, GPX and KML files as collections
• increased metadata about businesses from partner companies like InfoUSA, CitySearch, and Gayot.

Surprisingly, my favorite features of the new release don’t seem to have been mentioned. The removal of dual search boxes along with some of the other user interface niceties such as suggesting “Work” or “Home” when a user attempts to save a location are just great.

There was also a bug in the driving directions to my house from Redmond which was also fixed in this release. I never even got around to mentioning it to the team. Now that’s what I call service.  

Now playing: Lil Wayne - Go DJ (Remix) (feat. Slim Thug & Mannie Fresh)

Disclaimer: Although I work on the What’s New feed in Windows Live Spaces this should not be considered an announcement or precursor to an announcement of upcoming features of any Windows Live service.

I spend a lot of my time these days thinking about digital lifestyle aggregators such as Facebook and FriendFeed. One of the things I wonder about is how to make them more relevant to users as a way to stay connected to each other without seeming confusing, overwhelming or just plain spammy.

For instance, I look at the Facebook News Feed as the first significant implementation of this concept to hit the mainstream and I try to see what we can learn from their mistakes and where there is room for improvement. Below are two mistakes and one place I see room for improvement in the news feed as currently implemented by Facebook. The screenshot below is provided as a reference point.

Wall Posts

I’m now quite convinced that having wall posts show up in the news feed is a mistake. In general, Facebook already indulges in bad design by having a Wall-to-Wall posting which means that you can be viewing a friend’s wall and may only see one half of the conversation. So there is always a confusing loss of context when reading a wall on Facebook. This loss of context is exacerbated by adding wall posts to the news feed since now we not only have to deal with hearing one side of a conversation. Instead a user logs in and is confronted with a statement from the middle of a conversation, clicks through and only sees half the conversation, tries to click through to that and may not have access to both user’s walls.

More than once I’ve logged into Facebook and been confronted with wall posts that would have been embarassing to the posters if they realized that their banter on some person’s guestbook wall was being broadcast out of context to all their co-workers, their manager and even their VP/CxO via the news feed.

Application Installations

Although I don’t agree with Kara Swisher for criticizing Facebook applications as being mostly trivial time wasters instead of professional tools, I do agree with her that most apps on the site aren’t of value. This means that using up my screen real estate to tell me that a buddy has installed the Pink Ribbon application or the ProductPulse application is spam almost every single time you do it. One should also consider that Facebook limits the amount of updates from your friends they show in your news feed to ensure a good mix of updates. I suspect most users would gladly trade the slots taken up by notifications of application installs for more personally relevant updates from their social network.

The notifications about application installs showing up in the news feed is nice for developers but I question it’s value to users. Especially when you consider that Facebook applications already have ways to spread virally via application requests which has unfortunately led to Facebook Application Fatigue by their users.

Groups and Events

Today, a lot of groups on Facebook are there primarily as a way to declare affiliation as opposed to being an active community of users such as you’d find on sites like MSN Groups or Yahoo! Groups. I joined groups like I Dont care How Comfortable Crocs Are, You Look Like A Dumbass, I Am Fluent in Sarcasm, and If Wikipedia Says It, It Must Be True because I thought joining them would look funny on my friend’s news feeds as opposed to wanting to be part of these groups either as a lurker or as a regular discussion participant.

One question is why Facebook uses the news feed to drive user to user traffic but not user to group traffic besides the “Dare has joined People Who Always Have To Spell Their Names For Other People” which encourages people to join the group but doesn’t encourage them to participate in the group. It may be that they don’t want users creating online groups within the site like you find in services like MSN Groups or Yahoo! Groups or it could just be that their platform can’t support that scale of activity. I wonder…

The same questions apply for events as well. It would be cool if after I was invited to an event, I also got news feed updates via my news feed later on telling me if my friends were attending or that the event was getting lots of attendees which may influence my attendance. Again, you have to wonder why such obvious enhancements haven’t made it into their service.

Now playing: Three 6 Mafia - Late Nite Tip

Mary Jo Foley has a delightful post entitled Are all ‘open’ Web platforms created equal? where she wonders why there is more hype around the Facebook platform, Google’s muched hyped attempt to counter it on November 5th and other efforts that Anil Dash has accurately described as the new Blackbird as opposed to open API efforts from Microsoft. She posits two theories which are excerpted below

Who isn’t mentioned in any of these conversations? Microsoft. Is it because Microsoft hasn’t opened up its various Windows Live APIs to other developers? Nope. Microsoft announced in late April its plans for opening up and providing licensing terms for several of its key Windows Live APIs, including Windows Live Contacts, Windows Live Spaces Photo Control and Windows Live Data Protocols.

So why is Microsoft seemingly irrelevant to the conversation, when it comes to opening up its Web platform? There are a few different theories.

“I think the excitement about the Facebook platform stems from the fact that it addresses the problem of building publicity and distribution for a new application. Any developer can create an application for Facebook, and the social network will help propagate that application, exposing it to new users,” said Matt Rosoff, an analyst with Directions on Microsoft.

…

Microsoft, for its part, believes it is offering Web platform APIs the way that developers want, making them available under different business terms and permitting third parties to customize them inside their own sites, according to George Moore, General Manager of Windows Live. But Moore also acknowledges Microsoft has a different outlook in terms of which data it exposes via its APIs.

“Facebook gives you access to your social-graph (social-networking) data. We don’t do that. We have a gallery that allows users to extend Live Spaces,” Moore said.

Moore declined to comment on when or if Microsoft planned to allow developers to tap directly into user’s social-graph data like Facebook has done.

I see GeorgeM all the time, so I doubt he’ll mind if I clarify his statement above since it gives the wrong impression of our efforts given the context in which it was placed. If we go back to the definition of a social graph it’s clear that what is important is that it is a graph of user relationships not one that is tied to a particular site or service. From that perspective the Windows Live Contacts API which provides a RESTful interface to the contents of a user’s Windows Live address book complete with the list of tags/relationship types the user has applied to these contacts (e.g. “Family”, “Friends”, “Coworkers”, etc) as well as which of these contacts are the user’s buddies in Windows Live Messenger is a social graph API. 

On the other hand, this API does not give you access to the user’s Spaces friends list.  My assumption is that Mary Jo’s questions were specific to social networking sites which is why George gave that misleading answer. In addition, Yaron is fond of pointing out to me that the API is in alpha so there is still a lot that can change from now until we stamp it as v1. Until then, I’ll also decline to comment on any future plans.

As for the claim made by Matt Rosoff, I tend to agree with his assertion that the viral propagation of applications via the Facebook’s social graph is attractive to developers. However this attractiveness comes with the price of both the users and developers being locked in Facebook’s trunk.

I personally believe that the Web is the platform and this philosophy shines through in the API efforts at Microsoft. It may be that this is not as attractive to developers today as it should be but eventually the Web will win. Everyone who has fought the Web has lost. Facebook will not be an exception.

Now playing: Tony Yayo - I Know You Dont Love Me (feat. G-Unit)

Erick Schonfeld from TechCrunch writes in his post Windows Live SkyDrive Doubles Storage to 1GB, Still Can’t Keep Up With Gmail that

Microsoft doubled the online storage consumers can get for free in Windows Live SkyDrive. It’s hard to get excited about that when Gmail is already giving me 2.9 GB of storage, with more on the way—4GB by the end of the month, and 6GB by early January, according to one estimate.

You’d think that someone who works as a pseudo-journalist on a popular technology website would be able to tell the difference between an email service and a file storage service. You’d think he’d want to compare apples to apples and compare GMail’s 2.9 GB of Storage with Windows Live Hotmail’s 5 Gigabytes of Storage or compare the capabuilities of Microsoft’s SkyDrive with Google’s GDrive. 

Except Google hasn’t figured out how to ship GDrive for over 5 years so it would be an apples to vaporware comparison. 

Much love to my SkyDrive peeps on their new release. The champagne and ice cream yesterday was much appreciated. You can learn more about their release from the post Updates to Windows Live SkyDrive! on their team blog.  

Now playing: Foo Fighters - My Hero

Via Robert Scoble, I stumbled on Kara Swisher’s post entitled The Children’s Hour: Facebook Apps Are for Toddlers (There, We Said It) which has the following gem which is excerpted below

But, so far, as popular as those apps have become, what Zuckerberg and the widget-makers have wrought is mostly silly, useless and time-wasting and the kazillion users of these widgets are pretty much just acting like little children.

I never thought I would call the often frivolous AOL back in the day–very simply, a Neanderthal version of Facebook–a mature offering in comparison.

While I will admit when I am not chewing nails that a lot of these apps are somewhat fun, I can’t help but ask myself that lyric from the old Peggy Lee classic: “Is that all there is?”

This is like criticizing the Incredible Hulk for being green. You sound more ridiculous than what you are criticizing. The fact is Facebook is a site that was designed as an online version of the places where college students kill time in between classes. It’s a virtual place for “chilling in front of the library” or “maxing in front of the frat house”, so what do you expect from applications targetted at its users?

Next thing you know, Kara Swisher will be pointing out that most of the videos on YouTube aren’t worthy of being nominated for any academy awards in the short film category and that most of the widgets on MySpace are time wasting frivolities gadgets instead of professional tools.  .  

If you want a sophisticated widget platform that is intended for knowledge workers and business professionals, I’ve heard SalesForce AppExchange is exactly what you are looking for. Enjoy.

Now playing: Fabolous - Baby Don't Go (feat. Jermaine Dupri & T-Pain)

Two things I worked on over the past year or so shipped today. The first is Windows Live Events. You can learn more about in the post entitled Introducing Windows Live Events and new Windows Live Spaces updates by Chris Keating and Jay Fluegel which reads

Easily create a great-looking website for your next event
To offer you more ways to connect and share memories with the people you care about most, the team that brought you Spaces would love to hear your feedback on Windows Live Events, our new, free social event planning service.  With Events you can easily:

• Plan that next baby shower, birthday party, or family reunion
• Create a great-looking event invitation and website using one of over 100 fantastic templates
• Invite anyone with an e-mail address and track who’s coming
• Make your event unique with familiar customization features - choosing a friendly web address (like http://kates1stbirthday.events.live.com), using custom colors, fonts, and background images, or adding modules and Windows Live web gadgets
• Let guests and organizers share photos and stories before and after the event

Click on Events from the new navigation and then click Create event to get started!

My contribution to this release was working on modifying aspects of our contacts and storage platform to understand the concept of groups of users that can be treated as a single entity [especially with regards to joint ownership of objects, sharing and access control lists] instead of being centered on a single user. I expect that Windows Live Events will be just the first of many ways in which this capability will manifest itself across Windows Live.

Unfortunately, I didn't work on the final stage of getting the platform ready for the product to ship. Instead I went on to work on my next feature that shipped today while Ali took over working on the platform support for Windows Live Events including cleaning up my design hacks doing a better job of future proofing the design than I did. Mad props to Bob Bauman, Mike Newman, Jason Antonelli, John Kountz, Lalit Jina, Neel Malik, Mike Torres and everyone else who worked on this release across Windows Live. You guys rock.

The second thing I was a part of building which shipped today is the updated “What’s New” page in Windows Live Spaces which is also described in detail in the aforementioned post by Chris Keating and Jay Fluegel . Before you say anything, Yes, its re-design has been influenced by the Facebook News Feed feature. Below is a screen shot of the old design of the page from the previous release

and now contrast that with the new version of the page

I'm pretty jazzed about getting to work on this feature since it is something I've wanted do for a quite some time. A few years ago, I remember talking to Maya about building a “friends page” similar to the Live Journal friends page in MSN Spaces and at the time the response was that I was requesting that we merge the functionality of an RSS reader with a blogging/social networking site which was at cross purposes. In hindsight, I realize that although the idea was a good one, the implementation I was suggesting was kind of hokey. Then Facebook shipped the News Feed and it all clicked.

I worked with a lot of great folks on this feature. Paul Ming, Deepa Chandramouli, Rob Dolin, Vanesa Polo Dominguez, Jack Liu, Eric Melillo and a bunch of others who I may have failed to mention but still deserve lots of praise. This feature was the most fun I've had working in Windows Live. Not only did I get a deeper appreciation of designing for scalability but I also got to see what it is like to be responsible for components on the live site. All I need now is a pager and I'm good to go. :)

I'd be remiss in my duties if I didn't point out that in the second screen shot, the first item on the What's New page is less than 5 minutes old. If you use other systems that have similar features, you may have noticed a much longer delay than a few minutes from posting to showing up in your news feed. As the saying goes, a lot of effort went into making this look effortless.

I also noticed some initial feedback on this feature in the blog post by Jamie Thomson entitled new spaces home page where he writes

There's a lot of potential for this activity list given that it could capture any activity people commit using their Live ID. Every live property has the potential for being able to post activity on here so one day we may see notifications of:

• change of messenger status
• posting of photos on Live Space
• addition of gadgets to Live Space
• items placed for sale on Expo
• questions asked or answered on QnA
• collection shared from Live Maps
• video posted on MSN video
• changes to XBox gamer card
• changes to Zune Social (after it launches)
• items posted to the Live Gallery
• an event being planned
• purchased a song from Zune marketplace
• posts in MSN groups (soon to be Live Groups)
• posts to online forums (forums.microsoft.com)
• downloads of public files from Skydrive

Its all pretty good but let's be honest, this is basically a clone of of what Facebook already have. Given Facebook's popularity though Microsoft didn't really have a choice but to copy them. If Microsoft really want to differentiate themselves in this arena then one option would be to provide avenues for interacting with other online services such as Flickr, Twitter, Jaiku, Pownce,  etc... This list could then become an aggregator for all online activity and that's a pretty compelling scenario. One really quick win in this area would be to capture any blog entry that is posted from Live Writer, regardless of whether it is posted to Live Spaces or not.

Posting of photos already shows up on the "what's new" page. Downloads of files will likely never show up for privacy reasons, I'm sure you can guess why it may not be wise to broadcast what files you were downloading from shared folders to all your IM buddies and the people friends list if you think about it a little. As for the rest of the request, thanks for the feedback. We'll keep it in mind for future releases. 

PS: If you work for a Microsoft property that would like to show up on the "what's new" page, host it or just wanna plain chat about the feature then give me a shout if interested in the platform or holler at Rob if it's about the user experience.

I recently got an email from a developer about my post Thoughts on Amazon's Internal Storage System (Dynamo) which claimed that I seem to be romanticizing distributed systems that favor availability over consistency. He pointed out that although this sounds nice on paper, it places a significant burden on application developers. He is 100% right. This has been my experience in Windows Live and I’ve heard enough second hand to assume it is the experience at Amazon as well when it comes to Dynamo.

I thought an example of how this trade-off affects developers would be a useful excercise and may be of interest to my readers. The following example is hypothetical and should not be construed as describing the internal architectures of any production systems I am aware of.

Scenario: Torsten Rendelmann, a Facebook user in Germany, accepts a friend request from Dare Obasanjo who is a Facebook user in the United States.

The Distributed System: To improve the response times for users in Europe, imagine Facebook has a data center in London while American users a serviced from a Data Center in Palo Alto. To achieve this, the user database is broken up in a process commonly described as sharding. The question of if and how data is replicated across both data centers isn’t relevant to this example.

The application developer who owns the confirm_friend_request() method, will ideally want to write code that took the following form 

public void confirm_friend_request(user1, user2){

  begin_transaction();
  update_friend_list(user1, user2, status.confirmed); //palo alto
  update_friend_list(user2, user1, status.confirmed); //london
  end_transaction();

} 

Yay, distributed transactions. You have to love a feature that every vendor advises you not to use if you care about performance. So obviously this doesn’t work for a large scale distributed system where performance and availabilty are important.  

Things get particularly ugly when you realize that either data center or the specific server a user’s data is stored on could be unreachable for a variety of reasons (e.g. DoS attack, high seasonal load, drunk sys admin tripped over a power cord, hard drive failure due to cheap components, etc).

There are a number of options one can consider when availability and high performance are considered to be more important than data consistency in the above example. Below are three potential implementations of the code above each with it’s own set of trade offs.

OPTION I: Application developer performs manual rollback on error

public void confirm_friend_request_A(user1, user2){

 try{
   update_friend_list(user1, user2, status.confirmed); //palo alto 
 }catch(exception e){ 
  report_error(e); 
  return;
 }

 try{
  update_friend_list(user2, user1, status.confirmed); //london
 }catch(exception e) {
  revert_friend_list(user1, user2);
  report_error(e);
  return; 
 }

}

The problem here is that we don’t handle the case where revert_friend_list() fails. This means that Dare (user1) may end up having Torsten (user2) on his friend list but Torsten won’t have Dare on his friend list. The database has lied.

OPTION II: Failed events are placed in a message queue to be retried until they succeed.   

public void confirm_friend_request_B(user1, user2){

 try{
   update_friend_list(user1, user2, status.confirmed); //palo alto 
 }catch(exception e){ 
  report_error(e); 
  add_to_retry_queue(operation.updatefriendlist, user1, user2, current_time()); 
 }

 try{
  update_friend_list(user2, user1, status.confirmed); //london
 }catch(exception e) {
  report_error(e); 
  add_to_retry_queue(operation.updatefriendlist, user2, user1, current_time());  
 }

}

Depending on how long the error exists and how long it takes an item to sit in the message queue, there will be times when the Dare (user1) may end up having Torsten (user2) on his friend list but Torsten won’t have Dare on his friend list. The database has lied, again.

OPTION III: System always accepts updates but application developers may have to resolve data conflicts later. (The Dynamo approach)

/* update_friend_list always succeeds but may enqueue an item in message queue to try again later in the event of failure. This failure is not propagated to callers of the method.  */

public void confirm_friend_request_C(user1, user2){
   update_friend_list(user1, user2, status.confirmed); // palo alto
   update_friend_list(user2, user1, status.confirmed); //london 

}

/* get_friends() method has to reconcile results returned by get_friends() because there may be data inconsistency due to a conflict because a change that was applied from the message queue is contradictory to a subsequent change by the user.  In this case, status is a bitflag where all conflicts are merged and it is up to app developer to figure out what to do. */ 

  public list get_friends(user1){ 
      list actual_friends = new list();
      list friends = get_friends();  

      foreach (friend in friends){     

        if(friend.status == friendstatus.confirmed){ //no conflict
           actual_friends.add(friend);

        }else if((friend.status &= friendstatus.confirmed) 
                   and !(friend.status &= friendstatus.deleted)){

          // assume friend is confirmed as long as it wasn’t also deleted
          friend.status = friendstatus.confirmed;              
          actual_friends.add(friend); 
          update_friends_list(user1, friend, status.confirmed);

        }else{ //assume deleted if there is a conflict with a delete
          update_friends_list( user1, friend, status.deleted)
        }

      }//foreach

   return actual_friends;
}

These are just a few of the many approaches that can be implemented in such a distributed system to get around the performance and availability implications of using distributed transactions. The main problem with them is that in every single case, the application developer has an extra burden placed on his shoulders due to inherent fragility of distributed systems. For a lot of developers, the shock of this realization is akin to the shock of programming in C++ after using C# or Ruby for a couple of years. Manual memory management? Actually having to perform bounds checking arrays? Being unable to use decades old technology like database transactions?

The challenge in building a distributed storage system like BigTable or Dynamo is in balancing the need for high availability and performance while not building a system that encourages all sorts of insidious bugs to exist in the system by design.  Some might argue that Dynamo goes to far in the burden that it places on developers while there are others that would argue that it doesn’t go far enough.

In what camp do you fall?

Now playing: R. Kelly - Rock Star (feat. Ludacris & Kid Rock)

Yaron Goland has an entertaining post entitled Interoperability Wars - Episode 6 - Part 1 - Revenge of Babble about some of the philosophical discussions we’ve been having at work about the Atom Publishing Protocol (RFC 5023). The entire post is hilarious if you are an XML protocol geek and I recommend reading it. The following excerpt is a good starting point for another discussion about APP’s suitability as a general purpose editing protocol for the Web. Yaron writes

Emperor Babble: Excellent, Weasdel's death will serve us well in lulling the forces of interoperability into thinking they are making progress. Welcome Restafarian, it is time you understood your true place in my plans.

Luke Restafarian: Do what you want to me emperor, but the noble cause of interoperability will prevail!

The Emperor turns to the center of the chamber where a form, almost blinding in its clarity, takes shape:

GET /someuser/profile HTTP/1.1
host: example.com
content-type: application/xml

<profile xmlns="http://example.com">
   <professional>
      <workTitle>…</workTitle>
      …
   </professional>
   <personal>
      <spouseName>
         …
      </spouseName>
      …
   </personal>
   <clothingPreferences>
      <favoriteColors>
         <shirts>…</shirts>
         …
      </favoriteColors>
      …
   </clothingPreferences>
   …
</profile>

Darth Sudsy is momentarily taken aback from the appearance of the pure interoperable data while Luke Restafarian seems strengthened by it. The Emperor turns back to Luke and then speaks.

Emperor Babble: I see it strengthens you Restafarian, no matter. But look again at your blessed interoperability.

When the Emperor turns back to the form the form begins to morph, growing darker and more sinister:

GET /someuser/profileFeed HTTP/1.1
host: example.com
content-type: application/ATOM+xml

<feed xmlns="http://www.w3.org/2005/Atom">
   <title/>
   <updated>2000-01-01T00:00:00Z</updated>
   <author>
      <name/>
   </author>
   <id>http://www.example.com/someuser/profileFeed</id>
   <category scheme="http://example.com/categories" term="Profile"/>
   <entry>
      <title/>
      <id>http://www.example.com/someuser/profilesFeed/professional</id>
      <updated>2000-01-01T00:00:00Z</updated>
      <content type="Application/XML" xmlns:e="http://example.com">
         <workTitle>…</workTitle>
         …
      </content>
   </entry>
   <entry>
      <title/>
      <id>http://www.example.com/someuser/profilesFeed/personal</id>
      <updated>2000-01-01T00:00:00Z</updated>
      <content type="Application/XML" xmlns:e="http://example.com">
         <spouseName>
            …
         </spouseName>
         …
      </content>
   </entry>
   <entry>
      <title/>
      <id>http://www.example.com/someuser/profilesFeed/clothingPreferences</id>
      <updated>2000-01-01T00:00:00Z</updated>
      <content type="Application/XML" xmlns:e="http://example.com">
         <favoriteColors>
            <shirts>…</shirts>
            …
         </favoriteColors>
         …
      </content>
   </entry>
</feed>

Luke, having recognized the syntax, clearly expects to get another wave of strength but instead he feels sickly. The emperor, looking slightly stronger, turns from the form to look at Luke.

Luke Restafarian: What have you done? That structure is clearly taken from Princess Ape-Pea's system, she is a true follower of interoperability, I should be getting stronger but somehow it's making me ill.

Emperor Babble: You begin to understand young Restafarian. Used properly Princess Ape-Pea's system does indeed honor all the qualities of rich interoperability. But look more carefully at this particular example of her system. Is it not beautiful? Its needless complexity, its useless elements, its bloated form, they herald true incompatibility. No developer will be able to manually deal with such an incomprehensible monstrosity. We have taken your pure interoperable data and hidden it in a mud of useless scaffolding. Princess Ape-Pea and your other minions will accept this poison as adhering to your precious principles of interoperability but in point of fact by turning Princess Ape-Pea's system into a generic structured data manipulation protocol we have forced the data to contort into unnatural forms that are so hard to read, so difficult to understand that no programmer will ever deal with it directly. We will see no repeat of those damned Hit-Tip Knights building their own stacks in a matter of hours in order to enable basic interoperability. In this new world even the most trivial data visualizations and manipulations become a nightmare. Instead of simple transversals of hierarchical structures programmers will be forced into a morass of categories and artificial entry containers. Behold!

Yaron’s point is that since Atom is primarily designed for representing streams of microcontent, the only way to represent other types of data in Atom is to tunnel them as custom XML formats or proprietary extensions to Atom. At this point you’ve added an unnecessary layer of complexity.

The same thing applies to the actual Atom Publishing Protocol. The current design requires clients to use optimistic concurrency to handle conflicts on updates which seems like unnecessary complexity to push to clients as opposed to a “last update wins” scheme. Unfortunately, APP’s interaction model doesn’t support granular updates which means such a scheme isn’t supported by the current design. A number of APP experts have realized this deficiency as you can see from James Snell of IBM’s post entitled Beyond APP - Partial updates and John Panzer of Google’s post entitled RESTful partial updates: PATCH+Ranges.

A potential counter argument that can be presented when pointing these deficiencies of the Atom Publishing Protocol when used outside it’s core scenarios of microcontent publishing is that Google exposes all their services using GData which is effectively APP. This is true, but there are a couple of points to consider

1. Google had to embrace and extend the Atom format with several proprietary extensions to represent data that was not simply microcontent.
2. APP experts do not recommend embracing and extending the Atom format the way Google has done since it obviously fragments interoperability. See Joe Gregorio’s post entitled In which we narrowly save Dare from inventing his own publishing protocol and James Snell’s post entitled Silly for more details. 
3. Such practices leads to a world where we have applications that can only speak Google’s flavor of Atom. I remember when it used to be a bad thing when Microsoft did this but for some reason Google gets a pass.
4. Google has recognized the additional complexity they’ve created and now ship GData client libraries for several popular platforms which they recommend over talking directly to the protocol. I don’t know about you but I don’t need a vendor specific client library to process RSS feeds or access WebDAV resources, so why do I need one to talk to Google’s APP implementation?

It seems that while we weren’t looking, Google move us a step away from a world of simple, protocol-based interoperability on the Web to one based on running the right platform with the right libraries. 

Usually I wouldn’t care about whatever bad decisions the folks at Google are making with their API platform. However the problem is that it sends out the wrong message to other Web companies that are building Web APIs. The message that it’s all about embracing and extending Internet standards with interoperability being based on everyone running sanctioned client libraries instead of via simple, RESTful protocols is harmful to the Internet. Unfortunately, this harkens to the bad old days of Microsoft and I’d hate for us to begin a race to the bottom in this arena.

On the other hand, arguing about XML data formats and RESTful protocols are all variations of arguments about what color to paint the bike shed. At the end of the day, the important things are (i) building a compelling end user service and (ii) exposing compelling functionality via APIs to that service. The Facebook REST APIs are a clusterfuck of inconsistency while the Flickr APIs are impressive in how they push irrelevant details of the internals of the service into the developer API (NSIDs anyone?). However both of these APIs are massively popular.

From that perspective, what Google has done with GData is smart in that by standardizing on it even though it isn’t the right tool for the job, they’ve skipped the sorts of ridiculous what-color-to-paint-the-bike-shed discussions that prompted Yaron to write his blog post in the first place.

With that out of the way they can focus on building compelling Web services and exposing interesting functionality via their APIs. By the way, am I the last person to find out about the YouTube GData API?.

Now playing: DJ Khaled - I'm So Hood (feat. T-Pain, Trick Daddy, Rick Ross & Plies)

Jenna has uploaded pictures from our wedding weekend in Las Vegas and our honeymoon in Puerto Vallarta to her Windows Live space. Below are a couple of entry points into the photo stream.

These are the pictures that we took ourselves. The pics from the professionals capturing the wedding day and reception will show up in a couple of weeks. 

Now playing: Jagged Edge - Let's Get Married (remix) (feat. Run DMC)