Yesterday Twitter announced Fabric, a new mobile SDK for Android and iOS composed of four distinct pieces

• Crashlytics – an application analytics package that gives developers tools to measure how their apps are being used and measure app quality in the wild (i.e. crashes).
• Twitter Kit – an SDK that makes it add Twitter integration such as signing in with Twitter, embedding tweets or posting tweets from your app
• MoPub Kit – makes it easy to embed ads from Twitter’s ad network in your app so you can make money
• Digits – makes it easy for any app to build phone number based sign-in similar to what Skype Qik and WhatsApp have. This is quite frankly a game changer.

The response to this release I’ve seen online have swung between two extremes, fawning adoration from the tech press proclaiming that Twitter has moved beyond tweets into mobile services and skepticism from developers who don’t trust Twitter as represented in this tweet below

Did Twitter address in any way their platform trust issues today? Innovation in guaranteeing long term trustworthiness would be fascinating.

— kellan (@kellan) October 23, 2014

The root of this angst is Twitter’s tumultuous relationship with developers of Twitter clients which eventually led to their infamous quadrant of death post which effectively limited the growth of any app whose primary function was to be a replacement Twitter experience. This hurt many developers who had been working on Twitter reading experiences and in fact led to the CEO of Flipboard quitting Twitter’s board in disgust.

Thus it is a valid question for developers as to whether they can trust Twitter this time? The answer is Yes for a very simple reason. Twitter’s API moves in 2012 and yesterday’s announcements were borne from the same motives, to grow its primary business of selling ads tied to their mobile experiences. In 2012, they had to address the fact that their liberal exposure of their service via their API had created a situation where a huge slice of their user base were using the app through experiences Twitter could not effectively monetize. 

At the height of the 3rd party Twitter app boom almost half of their users were using official apps (42%) although that percentage dwindled as they stepped up their mobile app efforts and sent the message to app developers that they no longer wanted people to compete with them on providing mobile experiences.

Taking control of the primary user experience for Twitter was the smart business decision and is why they now  generate over a billion dollars a year as a business.

This brings us to Fabric. All four components aid Twitters core business of selling ads for mobile experiences.

• Twitter Kit increases engagement with Twitter by making it easy for users to consume and generate tweets from other apps without those apps being a threat to Twitter by becoming competing experiences.
• Digits allows Twitter to build a profile of users based on their phone number the same way Facebook builds a profile of users based on the apps and websites they visit that use Facebook Connect.
• Crashlytics + MoPub is the Trojan horse with a approach to Flurry which Yahoo acquired for $200 million. Crashlytics is a incredibly useful component that is valuable to all mobile apps since they all care about user behavior and crashes. Once you’re hooked on Crashlytics, it’s easier to upsell you to also using Twitters ad network and hence $$$.

All of these efforts help Twitter’s core business and it would be insanity for them to screw developers by abandoning them just as it would have been insanity for them to pursue an ad-based business model in a world where a huge chunk of their most active users were using 3rd party apps as their primary Twitter experience.

So go ahead, try out Fabric and judge it on its merits. I’m curious to hear what you think.

Now Playing: Chris Brown – Loyal (featuring Lil Wayne and Tyga)

A few weeks ago Dalton Caldwell, founder of imeem and Picplz, wrote a well received blog post titled What Twitter could have been where he laments the fact that Twitter hasn’t fulfilled some of the early promise developers saw in it as a platform. Specifically he writes

Perhaps you think that Twitter today is a really cool and powerful company. Well, it is. But that doesn’t mean that it couldn’t have been much, much more. I believe an API-centric Twitter could have enabled an ecosystem far more powerful than what Facebook is today. Perhaps you think that the API-centric model would have never worked, and that if the ad guys wouldn’t have won, Twitter would not be alive today. Maybe. But is the service we think of as Twitter today really the Twitter from a few years ago living up to its full potential? Did all of the man-hours of brilliant engineers, product people and designers, and hundreds of millions of VC dollars really turn into, well, this ?

His blog post struck a chord with developers which made Dalton follow it up with another blog post titled announcing an audacious proposal as well as launching the join.app.net. Dalton’s proposal is as follows

I believe so deeply in the importance of having a financially sustainable realtime feed API & service that I am going to refocus App.net to become exactly that. I have the experience, vision, infrastructure and team to do it. Additionally, we already have much of this built: a polished native iOS app, a robust technical infrastructure currently capable of handing ~200MM API calls per day with no code changes, and a developer-facing API provisioning, documentation and analytics system. This isn’t vaporware.

To manifest this grand vision, we are officially launching a Kickstarter-esque campaign. We will only accept money for this financially sustainable, ad-free service if we hit what I believe is critical mass. I am defining minimum critical mass as $500,000, which is roughly equivalent to ~10,000 backers.

As can be expected as someone who’s worked on software as both an API client and platform provider of APIs, I have a few thoughts on this topic. So let’s start from the beginning.

The Promise of Twitter Annotations

About two years ago, Twitter CEO Dick Costolo wrote an expansive “state of the union” style blog post about the Twitter platform. Besides describing the state of the Twitter platform at the time it also set forth the direction the platform intended to go in and made a set of promises about how Twitter saw its role relative to its developer ecosystem. Dick wrote

To foster this real-time open information platform, we provide a short-format publish/subscribe network and access points to that network such as www.twitter.com, m.twitter.com and several Twitter-branded mobile clients for iPhone, BlackBerry, and Android devices. We also provide a complete API into the functions of the network so that others may create access points. We manage the integrity and relevance of the content in the network in the form of the timeline and we will continue to spend a great deal of time and money fostering user delight and satisfaction. Finally, we are responsible for the extensibility of the network to enable innovations that range from Annotations and Geo-Location to headers that can route support tickets for companies. There are over 100,000 applications leveraging the Twitter API, and we expect that to grow significantly with the expansion of the platform via Annotations in the coming months.

There was a lot of excitement in the industry about Twitter Annotations both from developers as well as the usual suspects in the tech press like TechCrunch and ReadWriteWeb. Blog posts such as How Twitter Annotations Could Bring the Real-Time and Semantic Web Together show how excited some developers were by the possibilities presented by the technology. For those who aren’t familiar with the concept, annotations were supposed to be the way to attach arbitrary metadata to a tweet beyond the well-known 140 characters. Below is a screenshot from a Twitter presentation showing this concept visually

Although it’s been two years since the Twitter Annotations was supposed to begin testing this feature has never been released nor has it talked about by the Twitter Platform team in recent months. Many believe that Twitter Annotations will never be released due to a changing of the guard within Twitter which is hinted at in Dalton Caldwell’s What Twitter could have been post

As I understand, a hugely divisive internal debate occurred among Twitter employees around this time. One camp wanted to build the entire business around their realtime API. In this scenario, Twitter would have turned into something like a realtime cloud API company. The other camp looked at Google’s advertising model for inspiration, and decided that building their own version of AdWords would be the right way to go.

As you likely already know, the advertising group won that battle, and many of the open API people left the company.

It is this seeming change in direction that Dalton has seized on to create join.app.net.

Why Twitter Annotations was a Difficult Promise to Make

I have no idea what went on within Twitter but as someone who has built both platforms and API clients the challenges in delivering a feature such as Twitter Annotations seem quite obvious to me. A big problem when delivering a platform as part of an end user facing service is that there are often trade offs one has to make at the expense of the other. Doing things that make end users happy may end up ticking off people who’ve built businesses on your platform (e.g. layoffs caused by Google Panda update, companies losing customers overnight after launch of Facebook's timeline, etc). On the other hand, doing things that make developers happy can create suboptimal user experiences such as the “openness” of Android as a platform making it a haven for mobile malware.

The challenge with Twitter Annotations is that it threw user experience consistency out of the Window. Imagine that the tweet shown in the image above was created by Dare’s Awesome Twitter App which allows users to “attach” a TV episode from a source like Hulu or Netflix into the app before they tweet. Now in Dare’s Awesome Twitter App, the user sees their tweet an an inline experience where they can consume the video inline similar to what Twitter calls Expanded Tweets today. However the user’s friends who are using the Twitter website or other Twitter apps just see 140 characters. You can imagine that apps would then compete on how many annotations they supported and creating new interesting annotations. This is effectively what happened with RSS and a variety of RSS extensions with no RSS reader supporting the full gamut of extensions. Heck, I supported more RSS extensions in RSS Bandit in 2009 than Google Reader does today.

This cacophony of Annotations would have meant that not only would there no longer be such a thing as a consistent Twitter experience but Twitter itself would be on an eternal treadmill of supporting various annotations as they were introduced into the ecosystem by various clients and content producers.

Secondly, the ability to put arbitrary machine readable content in tweets would have made Twitter an attractive mechanism as a “free” publish-subscribe infrastructure of all manner of apps and devices. Instead of building a push notification system to communicate with client apps or devices, an enterprising developer could just create a special Twitter account and have all the end points connect to that. The notion of Twitter controlled botnets and the rise of home appliances connected to Twitter are all indicative that there is some demand for this capability. If this content not intended for humans ever becomes a large chunk of the data flowing through Twitter, how to monetize it will be extremely challenging. Charging people for using Twitter in this way isn’t easy since it isn’t clear how you differentiate a 20,000 machine botnet from a moderately popular Internet celebrity with a lot of fans who mostly lurk on Twitter.

I have no idea if Twitter ever plans to ship Annotations but if they do I’d be very interested to see how they would solve the two problems mentioned above.

Challenges Facing App.net as a Consumer Service

Now that we’ve talked about Twitter Annotations, let’s look at what App.net plans to deliver to address the demand that has yet to be fulfilled by the promise of Twitter Annotations. From join.app.net we learn the product that is intended to be delivered is

OK, great, but what exactly is this product you will be delivering?

As a member, you'll have a new social graph and real-time feed that you access from an App.net mobile application or website. At first, the user experience will be very similar to what Twitter was like before it turned into a media company. On a forward basis, we will focus on expanding our core experience by nurturing a powerful ecosystem based on 3rd-party developer built "apps". This is why we think the name "App.net" is appropriate for this service.

From a developer perspective, you will be able to read and write to a Twitter-like API. Developers behaving in good faith will have free reign to build alternate UIs, new business models of their own, and whatever they can dream up.

As this project progresses, we will be collaborating with you, our backers, while sharing and iterating screenshots of the app, API documentation, and more. There are quite a few technical improvements to existing APIs that we would like to see in the App.net API. For example, longer message character limits, RSS feeds, and rich annotations support.

In short it intends to be a Twitter clone with a more expansive API policy than Twitter. The key problem facing App.net is that as a social graph based services it will suffer from the curse of network effects. Social apps need to cross a particular threshold of people on the service before they are useful and once they cross that threshold it often leads to a “winner-take-all” dynamic where similar sites with smaller users bleed users to the dominant service since everyone is on the dominant service. This is how Facebook killed MySpace & Bebo and Twitter killed Pownce & Jaiku. App.net will need to differentiate itself more than just being “open version of popular social network”. Services like Status.net and Diaspora have tried and failed to make a dent with that approach while fresh approaches to the same old social graph and news feed like Pinterest and Instagram have grown by leaps and bounds.

A bigger challenge is the implication that it wants to be a paid social network. As I mentioned earlier, social graph based services live and die by network effects. The more people that use them the more useful they get for their members. Charging money for an online service is an easy way to reduce your target audience by at least an order of magnitude (i.e. reduce your target audience by at least 90%). As an end user, I don’t see the value of joining a Twitter-clone populated by people who could afford $50 a year as opposed to joining a free service like Facebook, Twitter or even Google+.

Challenges Facing App.net as a Developer Platform

As a developer it isn’t clear to me what I’m expected to get out of App.net. The question on how they came up with their pricing tiers is answered thusly

Additionally, there are several comps of users being willing to pay roughly this amount for services that are deeply valuable, trustworthy and dependable. For instance, Dropbox charges $10 and up per month, Evernote charges $5 and up per month, Github charges $7 and up per month.

The developer price is inspired by the amount charged by the Apple Developer Program, $99 . We think this demonstrates that developers are willing to pay for access to a high quality development platform.

I’ve already mentioned above that network effects are working against App.net as a popular consumer service. The thought of spending $50 a year to target less users than I can by building apps on Facebook or Twitter’s platforms doesn’t sound logical to me. The leaves using App.net as a low cost publish-subscribe mechanism for various apps or devices I deploy. This is potentially interesting but I’d need to get more data before I can tell just how interesting it could be. For example, there is a bunch of talk about claiming your Twitter handle and other things which makes it sound like developers only get a single account on the service. True developer friendliness for this type of service would include disclosure on how one could programmatically create and manage nodes in the social graph (aka accounts in the system).

At the end of the day although there is a lot of persuasive writing on join.app.net, there’s just not enough to explain why it will be a platform that will provide me enough value as a developer for me to part with my hard earned $50. The comparison to Apple’s developer program is rich though. Apple has given developers five billion reasons why paying them $99 a year is worth it, we haven’t gotten one good one from App.net.

That said, I wish Dalton luck on this project. Props to anyone who can pick himself up and continue to build new things after what he went through with imeem and PicPlz.

Now Playing: French Montana - Pop That (featuring Rick Ross, Lil Wayne and Drake)

Earlier this morning, Jeff Kunins posted the announcement that Messenger Connect is out of beta and available worldwide. Key excerpts from his post include

Today, we are pleased to announce that Messenger Connect is out of beta and available worldwide. We’ve gotten a great response so far: leading sharing syndicators ShareThis, AddThis, Gigya, and AddToThis have already made the Windows Live sharing badge available on more than 1 million websites (check it out now on Bing ).
…
Over 2500 developers gave us great feedback during the beta, helping us to refine and improve this release of Messenger Connect. Below is a quick summary, but for all the details check out this post from Angus on the Windows Live for Developers blog . Our focus with the release of Messenger Connect was to make it easier for partners to adopt, without compromising user privacy.

• Easier to check out –We made it faster and simpler for partners to try out Messenger Connect and determine if it would be useful for their sites. For example: you can try out the real time chat control without needing to write any code.
  
• Easier to adopt and integrate– Wereduced the effort needed for sites to implement Messenger Connect usefully and powerfully by providing new tools and sample code for ActivityStrea.ms template selectors and more.
  

A number of folks worked really hard behind the scenes to get us to this point and I’m glad to see what we’ve shipped today. I haven’t announced this on my blog yet but I recently took over as the Lead Program Manager responsible for our Messenger Connect and related platform efforts in Windows Live. If you’ve been a regular reader of my blog it shouldn’t be a surprise that I’ve decided to make working on building open web platforms my day job and not just a hobby I was interested in.

As Angus Logan says in his follow up blog post on the Windows Live for Developers blog; this is just the beginning. We’d love to get feedback from the community of developers on what we’ve released and the feedback we’ve gotten thus far has been immensely helpful. You can find us at http://dev.live.com/forums

PS: Since someone asked on Twitter, you can find out more about Messenger Connect by reading What is Messenger Connect?

Now Playing: Waka Flocka Flame - No Hands (featuring Roscoe Dash and Wale)

Earlier this morning, Ori Amiga posted Messenger across the Web on the Inside Windows Live blog. Key excerpts from his blog post include

Earlier today, John Richards and Angus Logan took the stage at The Next Web Conference in Amsterdam where they announced Messenger Connect – a new way for partners and developers to connect with Messenger. Messenger Connect allows web, Windows and mobile app developers to create compelling social experiences on their websites and apps by providing them with social promotion and distribution via Messenger.

…

Messenger Connect

Messenger Connect brings the individual APIs we’ve had for a long time (Windows Live ID, Contacts API, Messenger Web Toolkit , etc.) together in a single API that's based on industry standards and specifications ( OAuth WRAP , ActivityStrea.ms , PortableContacts ) and adds a number of new scenarios.

The new Messenger Connect provides our developer partners with three big things:

• Instantly create a user profile and social graph: Messenger user profile and social graph information allows our shared customers to easily sign-in and access their friends list and profile information. This allows our partners to more rapidly personalize their experiences, provides a ready-made social graph for customers to interact with, and provides a channel to easily invite additional friends to join in.
• Drive engagement directly through chat indirectly through social distribution: By enabling both real-time instant messaging conversations (chat) and feed-based sharing options for customers on their site, developers can drive additional engagement and usage of their experiences by connecting to the over 320 million Messenger customers worldwide.
• Designing for easy integration in your technical environment: We are delivering an API service that will expose a RESTful interface, and we’ll wrap those in a range of libraries (including JavaScript, .NET, and others). Websites and apps will be able to choose the right integration type for their specific scenario. Some websites prefer to keep everything at the presentation tier, and use JavaScript libraries when the user is present. Others may prefer to do server-side integration, so they can call the RESTful endpoints from back-end processes. We're aiming to provide the same set of capabilities across the API service and the libraries that we offer.

I’m really proud of the work that’s gone into building Messenger Connect. Although I was in some of the early discussions around it, I ducked out early to focus on the platform behind the new social view in Messenger and didn’t have much insight into the day to day of building the product. However I’ve got to say I love the way the project has turned out. I suspect a lot of web developers will as well.

Kudos to Ori and the rest of the team.

Now Playing: Ludacris - My Chick Bad (featuring Nicki Minaj)

In a move that was telegraphed by Fred Wilson’s (Twitter investor) post titled The Twitter Platform's Inflection Point where he criticized Twitter platform developers for “filling holes” in Twitter’s user experience, the Twitter team have indicated they will start providing official Twitter clients for various mobile platforms. There have been announcements of an official Blackberry client and the purchase of Tweetie so it can become the official iPhone client. The latter was announced in the blog post Twitter for iPhone excerpted below

Careful analysis of the Twitter user experience in the iTunes AppStore revealed massive room for improvement. People are looking for an app from Twitter, and they're not finding one. So, they get confused and give up. It's important that we optimize for user benefit and create an awesome experience.

We're thrilled to announce that we've entered into an agreement with Atebits (aka Loren Brichter) to acquire Tweetie, a leading iPhone Twitter client.

This has led to some anger on the part of Twitter client developers with some of the more colorful reactions being the creation of the Twitter Destroyed My Market Segment T-shirt and a somewhat off-color image that is making the rounds as representative of what Twitter means by “filling holes”.

As an end user and someone who works on web platforms, none of this is really surprising. Geeks consider having to wade through half a dozen Twitter clients before finding one that works for them a feature even though paradox of choice means that most people are actually happier with less choices not more. This is made worse by the fact that in the mobile world, this may mean paying for multiple apps until you find one that you’re happy with.

Any web company that cares about their customers will want to ensure that their experience is as simple and as pleasant as possible. Trusting your primary mobile experience to the generosity and talents of 3rd party developers means you are not responsible for the primary way many people will access your service. This loss of control isn’t great especially when the design direction you want to take your service in may not line up with what developers are doing in their apps. Then there’s the fact that forcing your users to make purchasing decisions before they can use your site conveniently on their phone isn’t a great first time experience either. 

I expect mobile clients are just the beginning. There are lots of flaws in the Twitter user experience that are due to Twitter’s reliance on “hole fillers” that I expect they’ll start to fill. The fact that I ever have to go to http://bit.ly as part of my Twitter workflow is a bug. URL shorteners really have no reason to exist in the majority of use cases except when Twitter is sending an SMS message. Sites that exist simply as image hosting services for Twitter like Twitpic and YFrog also seem extremely superflous especially when you consider that since only power users know about them not every Twitter user is figuring out how to use the service for image sharing. I expect this will eventually become a native feature of Twitter as well. Once Twitter controls the primary mobile clients for accessing their service, it’ll actually be easier for them to make these changes since they don’t have to worry about whether 3rd party apps will support Twitter image hosting vs. Twitpic versus rolling their own ghetto solution.

The situation is made particularly tough for 3rd party developers due to Twitter’s lack of a business model as Chris Dixon points out in his post Twitter and third-party Twitter developers

Normally, when third parties try to predict whether their products will be subsumed by a platform, the question boils down to whether their products will be strategic to the platform. When the platform has an established business model, this analysis is fairly straightforward (for example, here is my strategic analysis of Google’s platform).  If you make games for the iPhone, you are pretty certain Apple will take their 30% cut and leave you alone. Similarly, if you are a content website relying on SEO and Google Adsense you can be pretty confident Google will leave you alone. Until Twitter has a successful business model, they can’t have a consistent strategy and third parties should expect erratic behavior and even complete and sudden shifts in strategy.

So what might Twitter’s business model eventually be?  I expect that Twitter search will monetize poorly because most searches on Twitter don’t have purchasing intent .  Twitter’s move into mobile clients and hints about a more engaging website suggest they may be trying to mimic Facebook’s display ad model.

The hard question then is what opportunities will be left for developers on Twitter’s platform once the low hanging fruit has been picked by the company. Here I agree with frequent comments by Dave Winer and Robert Scoble, that there needs to be more metadata attached to tweets so that different data aggregation and search scenarios can be built which satisfy thousands of niches. I especially like what Dave Winer wrote in his post How Twitter can kill the Twitter-killers where he stated

Suppose Twitter wants to make their offering much more competitive and at the same time much more attractive to developers. Sure, as Fred Wilson telegraphed , some developers are going to get rolled over, esp those who camped out on the natural evolutionary path of the platform vendor. But there are lots of things Twitter Corp can do to create more opportunities for developers, ones that expand the scope of the platform and make it possible for a thousand flowers to bloom, a thousand valuable non-trivial flowers.

The largest single thing Twitter could do is open tweet-level metadata. If I want to write an app for dogs who tweet, let me add a "field" to a tweet called isDog, a boolean, that tells me that the author of the tweet is a dog. That way the dog food company who has a Twitter presence can learn that the tweet is from a dog, from the guy who's developing a special Twitter client just for dogs, even though Twitter itself has no knowledge of the special needs of dogs. We can also add a field for breed and age (in dog years of course). Coat type. Toy preference. A link to his or her owner. Are there children in the household?

I probably wouldn’t have used the tweeting dog example but the idea is sound. Location is an example of metadata that is added to tweets which can be used for interesting applications on top of the core news feed experience as shown by Twittervision and Bing's Twitter Maps. I think there’s an opportunity to build interesting things in this space especially if developers can invent new types of metadata without relying on Twitter to first bless new fields like they’ve had to do with location (although their current implementation is still inadequate in my opinion).

Over the next few months, Twitter will likely continue to encroach on territory which was once assumed to belong to 3rd party developers. The question is whether Twitter will replace these opportunities they’ve taken away with new opportunities or instead if they’ve simply used developers as a means to an end and now they are no longer useful?

Now Playing: Notorious B.I.G. - One More Chance

I initially planned to write up some detailed thoughts on the Google Wave video and the Google Wave Federation protocol. However the combination of the fact that literally millions of people have watched the video [according to YouTube] and I’ve had enough private conversations with others that have influenced my thinking that I’d rather not post something that makes it seem like I’m taking credit for the ideas of others. That said, I thought it would still be useful to share some of the most insightful commentary I’ve seen on Google Wave from various developer blogs.

Sam Ruby writes in his post Google Wave 

At one level, Google Wave is clearly a bold statement that “this is the type of application that every browser should be able to run natively without needing to resort to a plugin”.  At to give Google credit, they have been working relentlessly towards that vision, addressing everything from garbage collection issues , to enabling drag and drop of photos , to providing compelling content (e.g., Google Maps , GMail , and now Google Wave ).
…
But stepping back a bit, the entire and much hypedHTML5 interface is just a facade.  That’s not a criticism, in fact that’s generally the way the web works.  What makes Google Wave particularly interesting is that there is an API which operates directly on the repository.  Furthermore, you can host your own server, and such servers federate using XMPP.

These servers are not merely passive, they can actively interact with processes called “robots” using HTTP (More specifically, JSON-RPC over POST).  Once invoked, these robots have access to a full range of operations ( Java , Python ).  The Python library implementation looks relatively straightforward, and would be relatively easy to port to, say Ruby.

This dichotomy pointed out by Sam is very interesting. One the one hand, there is the Google Wave web application which pushes the boundaries of what it means to be a rich web application that simply uses Javascript and the HTML DOM. This is a companion step in Google’s transition to taking an active role in the future of building Web applications where previous steps have included Google representatives drafting the HTML 5 specification, Google Gears and Google Chrome. However where things get interesting is that the API makes it possible to build alternate client applications (e.g. a .NET Wave client written in C#) and even build services that interact with users regardless of which Wave client they are using.

Joe Gregorio has more on these APIs in his blog post Wave Protocol Thoughts where he writes

There are actually 3 protocols and 2 APIs that are used in Wave:

• Federation (XMPP)
• The robot protocol (JSONRPC)
• The gadget API (OpenSocial)
• The wave embed API (Javascript)
• The client-server protocol (As defined by GWT)

The last one in that list is really nothing that needs to be, or will probably ever be documented, it is generated by GWT and when you build your own Wave client you will need to define how it talks to your Wave server. The rest of the protocols and APIs are based on existing technologies.

The robot protocol looks very easy to use, here is the code for an admittedly simple robot . Now some people have commented that Wave reminds them of Lotus Notes, and I'm sure with a little thought you could extend that to Exchange and Groove. The difference is that the extension model with Wave is events over HTTP, which makes it language agnostic, a feature you get when you define things in terms of protocols. That is, as long as you can stand up an HTTP server and parse JSON, you can create robots for Wave, which is a huge leap forward compared to the extension models for Notes, Exchange and Groove, which are all "object" based extension models. In the "object" based extension model the application exposes "objects" that are bound to locally that you manipulate to control the application, which means that your language choices are limited to those that have bindings into that object model.

As someone’s whose first paying job in the software industry was an internship where I had to write Outlook automation scripts to trigger special behaviors when people sent or modified Outlook task requests, I can appreciate the novelty of moving away from a programming model based on building a plugin in an application’s object model and instead building a Web service and having the web application notify you when it is time to act which is the way the Wave robot protocol works. Now that I’ve been exposed to this idea, it seems doubly weird that Google also shipped Google Apps Script within weeks of this announcement. 

Nick Gall writes in his post My 2¢ on Google Wave: WWW is a Unidirectional Web of Published Documents -- Wave is a bidirectional Web of Instant Messages that

Whether or not the Wave client succeeds, Wave is undoubtedly going to have a major impact on how application designers approach web applications. The analogy would be that even if Google Maps had "failed" to become the dominant map site/service, it still had major impact on web app design.

I suspect this as well. Specifically I have doubts about the viability of the communications paradigm shift that Google Wave is trying to force taking hold. On the other hand, I’m sure there are thousands of Web developers out there right now asking themselves "would my app be better if users could see each other’s edits in real time?","should we add a playback feature to our service as well" [ed note - wikipedia could really use this] and "why don’t we support seamless drag and drop in our application?". All inspired by their exposure to Google Wave.

Finally, I've ruminated publicly that I see a number of parallels between Google Wave and the announcement of Live Mesh. The one interesting parallel worth calling out is that both products/visions/platforms are most powerful when there is a world of different providers each exposing their data types to one or more of these rich user applications (i.e. a Mesh client or Wave client). Thus far I think Google has done a better job than we did with Live Mesh in being very upfront about this realization and evangelizing to developers that they participate as providers. Of course, the proof will be in the pudding in a year or so when we see just how many services have what it takes to implement a truly interoperable federated provider model for Google Wave.

In my previous post I talk about adding support for reading and commenting on your Facebook news feed in RSS Bandit. This functionality is made possible by the recently announced Facebook Open Stream API. As I worked through the code in my few free moments I was alternately impressed and frustrated by the Open Stream API. On the one hand, the functionality the API provides is greatly enabling and has already unleashed a bounty of innovation as evidenced by the growing number of applications for interacting with your Facebook news feed on the desktop (e.g. Seesmic, Tweetdeck, bdule, etc). On the other hand, figuring out there are a few quirks of the API that make the web developer in me cringe and the desktop developer want to beg for mercy.

Below are my opinions on the Open Stream API, the purpose of sharing is so other developers who plan to use the API may avoid some of the pitfalls I did and also to share feedback with my peers at Facebook and elsewhere on best practices in providing activity stream APIs. 

Good: Lots of options for getting at the data 

Most API calls at Facebook have two entry points. You can either call a “REST-like” URL endpoint via HTTP GET or perform a SQL-like query via FQL on a generic end point. For interacting with the Facebook news feed, you have stream.get and stream.getComments as the REST-like methods for accessing the news feed and a comment thread for a particular feed item respectively.  With FQL, you can perform queries against the stream (FQL) and comment (FQL) to get the same results.

Both mechanisms give you the option of getting back the data as XML or JSON. Below are example of what HTTP requests to retrieve your news feed look like using both approaches 

stream.get REQUEST:
http://api.facebook.com/restserver.php?v=1.0&method=stream.get&format=XML&viewer_id={0}&session_key={1}&api_key={2}&call_id={3}&sig={4}

FQL REQUEST:
http://api.facebook.com/restserver.php?v=1.0&method=fql.query&format=XML&call_id={0}&session_key={1}&api_key={2}&sig={3}&query={4}
FQL QUERY:
select post_id, source_id, created_time, actor_id, target_id, app_id, message, attachment, comments, likes, permalink, attribution, type from stream where filter_key in (select filter_key FROM stream_filter where uid = {0} and type = 'newsfeed') and created_time >= {1} order by created_time desc limit 50

I ended up using stream.get over FQL because the only benefit I see to using FQL is being able to filter some of the result fields or combine data from multiple tables, neither of which I needed in this instance. I chose XML as the output format over JSON because I transform the results of the Facebook API request into an Atom feed and then hand it down to the regular Atom feed parsing code in RSS Bandit. It was much easier for me to handle this transformation using XSLT over XML results as opposed to procedural code over a JSON result set.

There’s also a third mechanism for getting news feed data out of Facebook that would have been perfect for my needs. You can access the news feed using the nascent activity streams standard which returns the data as an Atom feeds with certain extensions specific to social network activity stream. I couldn’t get this to work probably due to user error on my part (more on this later) but if I had, the structure of the GET request would have been along the lines of

Activity Streams REQUEST:
http://www.facebook.com/activitystreams/feed.php?source_id={0}&app_id={1}&session_key={2}&sig={3}&v=0.7&read&updated_time={4}

If not for the initial problems I had figuring out which parameters to pass to APIs and the concern about building on an API that isn’t yet at version 1.0, the Activity Stream API would have been perfect for my needs.

Kudos to the Facebook team for providing such a rich and varied set of options for getting the news feed out of their system. There’s something for every temperament.

Bad: Misuse of HTTP status codes

The Facebook API documentation describes the platform as “REST-like” and not RESTful. They sure weren’t kidding. For the most part, when I’ve discussed the problems with APIs that aren’t completely RESTful the negatives to such an approach have seemed aesthetic in nature as opposed to being practical problems. Below are some of the practical problems I faced because the Facebook APIs do not use HTTP status codes in a manner consistent with the rest of the Web.

In HTTP, there are existing error codes which clients can interpret in a consistent manner and provide consistent feedback to users when they occur. Now consider this excerpt from the documentation on using Facebook’s activity stream API

Response Codes

Like most HTTP responses, Facebook Activity Streams responses include a response header, which always includes a traditional response code and a short response message. The supported response codes include:

• 200 Code provided whenever the Facebook servers were able to accommodate the request and provide a response.
• 304 Code provided whenever the request header included If-Modified-Since and no new posts have been generated since the specified time.
  Note: Code 304 will never be returned if If-Modified-Since isn't included in the request header.
• 401 Code provided whenever the URL omits one or more of the required parameters.
• 403 Code provided whenever the URL is syntactically valid, but the user hasn't granted the required extended permission.
• 404 Code provided whenever the URL is syntactically valid, but the signature is incorrect, or the session key is invalid.

You might notice that an HTTP 401 is used to indicate that the request is improperly formed. However, let’s see what RFC 2616 has to say about the 401 status code as well as how to communicate badly formed arguments

400 Bad Request

The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.

401 Unauthorized

The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8 ). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication"

According to the HTTP specification excerpted above, the status code to return on missing parameters should be 400 not 401. The practical problem with returning HTTP 401 in this case is that applications like RSS Bandit may have code paths that prompt the user to check or re-enter their credentials because an authentication error has occurred. We now have to special case getting a 401 from Facebook’s servers versus any other server on the Web. Thankfully, this error should be limited to development time unless Facebook changes their API in a backwards incompatible manner by requiring new parameters to their methods.

A bigger problem is that Facebook returns HTTP 200 which traditionally means success in regular occurring error conditions. Specifically, when a user’s session key expires a successful response is sent containing an error document. In RSS Bandit, we already have a processing pipeline that hands off successful responses containing an XML document to the RSS/Atom parsing layer. With Facebook’s behavior I had two choices

1. Modify the RSS/Atom parsing layer to understand Facebook error documents and then kick back an error up to the user interface asking the user to re-enter their credentials. This was particularly hacky because that layer doesn’t really have a connection to the main UI thread nor should it.

2. Pre-process the XML document before handing it to the RSS/Atom parsing layer. This implies an intermediate step in between dispatching on the response status and actually processing the document.

Option #2 proved more palatable since there was already an intermediate step needed to transform the results of stream.get to an Atom feed. If I’d used the Activity Streams API as was my initial plan then the decision may have been harder to make.

Good: Well thought out model for authorizing access to user data

A number of application platforms work by giving a user all-or-nothing access to the user’s data. My favorite example of this bad practice is Twitter which for a long time made this situation even worse by requiring the applications to collect the person’s username and password. The controversy around Twply earlier this year shows exactly why giving applications more access to a user’s data than they need is bad. Twply only needed access to a user’s @replies but given that there was no way to only give it access to just that aspect of a user’s Twitter data, it also got the ability to post tweets on their behalf and did so in a spammy manner.

The Facebook API has a notion of extended permissions which are access rights that require special opt-in from the user given that the data or functionality is sensitive and can be abused. The current set of extended permissions in the Facebook API is provided below

Bad: Too many prompts for desktop applications

Although I’ve praised the extended permissions model, it currently leads to a cumbersome experience for desktop applications. Installing Facebook desktop applications like bdule or Facebook for Adobe Air requires running through three separate permissions screens. The user has to login, then grant the read_stream extended permission followed by the publish_stream extended permission. Granted, the latter two only need to be done once but they still affect the out of box experience fairly negatively in my opinion.

In RSS Bandit, I’ve attempted to reduce this by delaying the prompt for publish_stream permission until the first time a user tries to comment on a news feed item from within the application. Streamlining this experience will be a boon for application developers who want the entire experience to be smooth and painless. The documentation on the documentation the Open Streams API page states that there are options for streamlining these requests but they only apply to Web applications not desktop apps.

Bad: Plethora of application identifiers and authentication requirements is a stumbling block for beginners

The official documentation on the Facebook API doesn’t do a good job of connecting all the dots when it comes to understanding how to make calls to the service. For example, when you read the documentation for stream.get it is not obvious from that page what endpoint to make requests to OR that every Facebook API call has a set of required parameters beyond the ones listed on that page. In fact, I was stumped for about a week or so trying to figure out the magical incantations to get the right set of parameters for API calls and the right data to put in these parameters until I stumbled on a Facebook forum post creating the 'sig' parameter which solved my problems. I believe I once saw this information in the Facebook API documents but after ten minutes of searching just now I can’t seem to find it so perhaps it was in my imagination.

Part of the problem is the varied number of identifiers that you have to keep straight as an application developer including your

• application ID
• API key
• application secret
• session key
• client secret

The fact that various APIs take different combinations of the above lead to more than one confusing moment for me. Eventually I figured it out but I felt like I was being hazed as I was going through the process.

Over the past few days I've been mulling over the recent news from Yahoo! that they are building a Facebook-like platform based on OpenSocial. I find this interesting given that a number of people have come to the conclusion that Facebook is slowly killing it's widget platform in order to replace it with Facebook Connect.

The key reason developers believe Facebook is killing of its platform is captured in Nick O'Neill's post entitled Scott Rafer: The Facebook Platform is Dead which states

When speaking at the Facebook developer conference today in Berlin, Scott Rafer declared that Facebook platform dead. He posted statistics including one that I posted that suggests Facebook widgets are dead . Lookery’s own statistics from Quantcast suggest that their publisher traffic has been almost halved since the new site design was released. Ultimately, I think we may see an increase in traffic as users become educated on the new design but there is no doubt that developers were impacted significantly.

So what is Scott’s solution for developers looking to thrive following the shift to the new design? Leave the platform and jump on the Facebook Connect opportunity.

The bottom line is that by moving applications off of the profile page in their recent redesign, Facebook has reduced the visibility of these application thus reducing their page views and their ability to spread virally. Some may think that the impact of these changes is unforeseen, however the Facebook team is obsessive about testing the impact of their UX changes so it is extremely unlikely that they aren't aware that the redesign would negatively impact Facebook applications.

The question to ask then is why Facebook would knowingly damaging a platform which has been uniformly praised across the industry and has had established Web players like Google and Yahoo! scrambling to deploy copycat efforts? Alex Iskold over at ReadWriteWeb believes he has the answer in his post Why Platforms Are Letting Us Down - And What They Should Do About It which contains the following excerpt

When the Facebook platform was unveiled in 2007, it was called genius. Never before had a company in a single stroke enabled others to tap into millions of its users completely free. The platform was hailed as a game changer under the famous mantra "we built it and they will come". And they did come, hundreds of companies rushing to write Facebook applications. Companies and VC funds focused specifically on Facebook apps.

It really did look like a revolution, but it didn't last . The first reason was that Facebook apps quickly arranged themselves on a power law curve. A handful of apps (think Vampires, Byte Me and Sell My Friends) landed millions of users, but those in the pack had hardly any. The second problem was, ironically, the bloat. Users polluted their profiles with Facebook apps and no one could find anything in their profiles. Facebook used to be simple - pictures, wall, friends. Now each profile features a zoo of heterogenous apps, each one trying to grab the user's attention to take advantage of the network effect. Users are confused.

Worst of all, the platform had no infrastructure to monetize the applications. When Sheryl Sandberg arrived on the scene and looked at the balance sheet, she spotted the hefty expense that was the Facebook platform. Trying to live up to a huge valuation isn't easy, and in the absense of big revenues people rush to cut costs. Since it was both an expense and users were confused less than a year after its glorious launch, Facebook decided to revamp its platform.

The latest release of Facebook , which was released in July, makes it nearly impossible for new applications to take advantage of the network effect. Now users must first install the application, then find it under the application menu or one of the tabs, then check a bunch of boxes to add it to their profile (old applications are grand-daddied in). Facebook has sent a clear message to developers - the platform is no longer a priority.

Alex's assertion is that after Facebook looked at the pros and cons of their widget platform, the company came to the conclusion that the platform was turning into a cost center instead of being away to improve the value of Facebook to its users. There is evidence that applications built on Facebook's platform did cause negative reactions from its users. For example, there was the creation of the "This has got to stop…pointless applications are ruining Facebook" group which at its height had half a million Facebook users protesting the behavior of Facebook apps. In addition, the creation of Facebook's Great Apps program along with the guiding principles for building Facebook applications implies that the Facebook team realized that applications being built on their platform typically don't have their users best interests at heart.

This brings up the interesting point that although there has been a lot of discussion on how Facebook apps make money there haven't been similar conversations on how the application platform improves Facebook's bottom line. There is definitely a win-win equation when so-called "great apps" like iLike and Causes, which positively increase user engagement, are built on Facebook's platform. However there is also a long tail of applications that try their best to spread virally at the cost of decreasing user satisfaction in the Facebook experience. These dissatisfied users likely end up reducing their usage of Facebook thus actually costing Facebook users and page views. It is quite possible that the few "great apps" built on the Facebook platform do not outweigh the amount of not-so-great apps built on the platform which have caused users to protest in the past. This would confirm Alex Iskold's suspicions about why Facebook has started sabotaging the popularity of applications built on its platform and has started emphasizing partnerships via Facebook Connect.

A similar situation has occurred with regards to the Java platform and Sun Microsystems. The sentiment is captured in a Javaworl article by Josh Fruhlinger entitled Sun melting down, and where's Java? which contains the following excerpt

one of the most interesting things about the coverage of the company's problems is how Java figures into the conversation, which is exactly not at all. In most of the articles, the word only appears as Sun's stock ticker; the closest I could find to a mention is in this AP story , which notes that "Sun's strategy of developing free, 'open-source' software and giving it away to spur sales of its high-end computer servers and support services hasn't paid off as investors would like." Even longtime tech journalist Ashlee Vance, when gamely badgering Jon Schwartz for the New York Time about whether Sun would sell its hardware division and focus on software , only mentions Solaris and MySQL in discussing the latter.

Those in the Java community no doubt believe that Java is too big to fail, that Sun can't abandon it because it's too important, even if it can't tangibly be tied to anything profitable. But if Sun's investors eventually dismember the company to try to extract what value is left in it, I'm not sure where Java will fit into that plan.

It is interesting to note that after a decade of investment in the Java platform, it is hard to point to what concrete benefits Sun has gotten from being the originator and steward of the Java platform and programming language. Definitely another example of a platform that may have benefited applications built on it yet which didn't really benefit the platform vendor as expected.

The lesson here is that building a platform isn't just about making the developers who use the platform successful but also making sure that the platform itself furthers the goals of its developers in the first place.

Now Playing: Kardinal Offishall - Dangerous (Feat. Akon)

Just because you aren't attending Microsoft's Professional Developer Conference doesn't mean you can't follow the announcements. The most exciting announcement so far [from my perspective] has been Windows Azure which is described as follows from the official site

The Azure™ Services Platform (Azure) is an internet-scale cloud services platform hosted in Microsoft data centers, which provides an operating system and a set of developer services that can be used individually or together. Azure’s flexible and interoperable platform can be used to build new applications to run from the cloud or enhance existing applications with cloud-based capabilities. Its open architecture gives developers the choice to build web applications, applications running on connected devices, PCs, servers, or hybrid solutions offering the best of online and on-premises.

Azure reduces the need for up-front technology purchases, and it enables developers to quickly and easily create applications running in the cloud by using their existing skills with the Microsoft Visual Studio development environment and the Microsoft .NET Framework. In addition to managed code languages supported by .NET, Azure will support more programming languages and development environments in the near future. Azure simplifies maintaining and operating applications by providing on-demand compute and storage to host, scale, and manage web and connected applications. Infrastructure management is automated with a platform that is designed for high availability and dynamic scaling to match usage needs with the option of a pay-as-you-go pricing model. Azure provides an open, standards-based and interoperable environment with support for multiple internet protocols, including HTTP, REST, SOAP, and XML.

It will be interesting to read what developers make of this announcement and what kind of apps start getting built on this platform. I'll also be on the look out for any in depth discussions on the platform, there is lots to chew on in this announcement.

For a quick overview of what Azure means to developers, take a look at Azure for Business and Azure for Web Developers. 

Now Playing: Guns N' Roses - Welcome to the Jungle

Tim Bray has a thought provoking post on embracing cloud computing entitled Get In the Cloud where he brings up the problem of vendor lock-in. He writes

Tech Issue · But there are two problems. The small problem is that we haven’t quite figured out the architectural sweet spot for cloud platforms. Is it Amazon’s EC2/S3 “Naked virtual whitebox” model? Is it a Platform-as-a-service flavor like Google App Engine? We just don’t know yet; stay tuned.

Big Issue · I mean a really big issue: if cloud computing is going to take off, it absolutely, totally, must be lockin-free. What that means if that I’m deploying my app on Vendor X’s platform, there have to be other vendors Y and Z such that I can pull my app and its data off X and it’ll all run with minimal tweaks on either Y or Z.

At the moment, I don’t think either the Amazon or Google offerings qualify.

Are we so deranged here in the twenty-first century that we’re going to re-enact, wide-eyed, the twin tragedies of the great desktop-suite lock-in and the great proprietary-SQL lock-in? You know, the ones where you give a platform vendor control over your IT budget? Gimme a break.

I’m simply not interested in any cloud offering at any level unless it offers zero barrier-to-exit.

Tim's post is about cloud platforms but I think it is useful to talk about avoiding lock-in when taking a bet on cloud based applications as well as when embracing cloud based platforms. This is especially true when you consider that moving from one application to another is a similar yet smaller scoped problem compared to moving from one Web development platform to another.

So let's say your organization wants to move from a cloud based office suite like Google Apps for Business to Zoho. The first question you have to ask yourself is whether it is possible to extract all of your organization's data from one service and import it without data loss into another. For business documents this should be straightforward thanks to standards like ODF and OOXML. However there are points to consider such as whether there is an automated way to perform such bulk imports and exports or whether individuals have to manually export and/or import their online documents to these standard formats. Thus the second question is how expensive it is for your organization to move the data. The cost includes everything from the potential organizational downtime to account for switching services to the actual IT department cost of moving all the data. At this point, you then have to weigh the impact of all the links and references to your organization's data that will be broken by your switch. I don't just mean links to documents returning 404 because you have switched from being hosted at google.com to zoho.com but more insidious problems like the broken experience of anyone who is using the calendar or document sharing feature of the service to give specific people access to their data. Also you have to ensure that email that is sent to your organization after the switch goes to the right place. Making this aspect of the transition smooth will likely be the most difficult part of the migration since it requires more control over application resources than application service providers typically give their customers. Finally, you will have to evaluate which features you will lose by switching applications and ensure that none of them is mission critical to your business.

Despite all of these concerns, switching hosted application providers is mostly a tractable problem. Standard data formats make data migration feasible although it might be unwieldy to extract the data from the service. In addition, Internet technologies like SMTP and HTTP all have built in ways to handle forwarding/redirecting references so that they aren't broken. However although the technology makes it possible, the majority of hosted application providers fall far short of making it easy to fully migrate to or away from their service without significant effort.

When it comes to cloud computing platforms, you have all of the same problems described above and a few extra ones. The key wrinkle with cloud computing platforms is that there is no standardization of the APIs and platform technologies that underlie these services. The APIs provided by Amazon's cloud computing platform (EC2/S3/EBS/etc) are radically different from those provided by Google App Engine (Datastore API/Python runtime/Images API/etc). For zero lock-in to occur in this space, there need to be multiple providers of the same underlying APIs. Otherwise, migrating between cloud computing platforms will be more like switching your application from Ruby on Rails and MySQL to Django and PostgreSQL (i.e. a complete rewrite).

In response to Tim Bray's post, Dewitt Clinton of Google left a comment which is excerpted below

That's why I asked -- you can already do that in both the case of Amazon's services and App Engine. Sure, in the case of EC2 and S3 you'll need to find a new place to host the image and a new backend for the data, but Amazon isn't trying to stop you from doing that. (Actually not sure about the AMI format licensing, but I assumed it was supposed to be open.)

In App Engine's case people can run the open source userland stack (which exposes the API you code to) on other providers any time the want, and there are plenty of open source bigtable implementations to chose from. Granted, bulk export of data is still a bit of a manual process, but it is doable even today and we're working to make it even easier.

Ae you saying that lock-in is avoided only once the alternative hosts exist?

But how does Amazon or Google facilitate that, beyond getting licensing correct and open sourcing as much code as they can? Obviously we can't be the ones setting up the alternative instances. (Though we can cheer for them, like we did when we saw the App Engine API implemented on top of EC2 and S3.)

To Doug Cutting's very good point, the way Amazon and Google (and everyone else in this space) seem to be trying to compete is by offering the best value, in terms of reliability (uptime, replication) and performance (data locality, latency, etc) and monitoring and management tools. Which is as it should be.

Although Dewitt is correct that Google and Amazon are not explicitly trying to lock-in customers to their platform, the fact is that today if a customer has heavily invested in either platform then there isn't a straightforward way for customers to extricate themselves from the platform and switch to another vendor. In addition there is not a competitive marketplace of vendors providing standard/interoperable platforms as there are with email hosting or Web hosting providers.

As long as these conditions remain the same, it may be that lock-in is too strong a word describe the situation but it is clear that the options facing adopters of cloud computing platforms aren't great when it comes to vendor choice.

Now Playing: Britney Spears - Womanizer

Werner Vogels, CTO of Amazon, writes in his blog post Expanding the Cloud: Microsoft Windows Server on Amazon EC2 that

With today's announcement that Microsoft Windows Server is available on Amazon EC2 we can now run the majority of popular software systems in the cloud. Windows Server ranked very high on the list of requests by customers so we are happy that we will be able to provide this.

One particular area that customers have been asking for Amazon EC2 with Windows Server was for Windows Media transcoding and streaming. There is a range of excellent codecs available for Windows Media and there is a large amount of legacy content in those formats. In past weeks I met with a number of folks from the entertainment industry and often their first question was: when can we run on windows?

There are many different reasons why customers have requested Windows Server; for example many customers want to run ASP.NET websites using Internet Information Server and use Microsoft SQL Server as their database. Amazon EC2 running Windows Server enables this scenario for building scalable websites. In addition, several customers would like to maintain a global single Windows-based desktop environment using Microsoft Remote Desktop, and Amazon EC2 is a scalable and dependable platform on which to do so.

This is great news. I'm starting a month long vacation as a precursor to my paternity leave since the baby is due next week and was looking to do some long overdue hacking in-between burping the baby and changing diapers. My choices were

• Facebook platform app
• Google App Engine app
• EC2/S3/EBS app

The problem with Amazon was the need to use Linux which I haven't seriously messed with since my college days running SuSe. If I could use Windows Server and ASP.NET while still learning the nuances of EC2/S3/EBS that would be quite sweet.

I wonder how who I need to holler at to get in the Windows Server on EC2 beta? Maybe Derek can hook me up. Hmmmmm.

Now Playing: Lil Wayne - Best Rapper Alive [Explicit]

I'm in the market for a new phone and I've been considering getting an iPhone 3G to replace my AT&T Tilt (aka HTC Kaiser). The Tilt is a great PDA (thanks to Windows Mobile 6) and I love the slide out QWERTY keyboard. My main problems with it are the relatively huge physical size, small amount of storage space and needing two hands if I want to send email or text messages.

Although I've recently seen a lot of hype around Google's Android operating system and T-Mobile G1 (aka the HTC Dream), I haven't had any interest in it since it has no support for integrating with Microsoft Exchange which is the only reason I want a smart phone in the first place. However I have found it interesting that a lot of recent blog posts about the iPhone are about how it is in a weak position against Android because Google's open approach will trump Apple's closed approach with regards to their developer platform.

A typical example of this trend in iPhone coverage is Antionio Cangiano's blog post entitled Don't Alienate Developers which is excerpted below

Apple, a company that is generally considered far from “sinister” or “evil”, on the other hand, is trying their best to alienate developers.
…
Their first idiotic move was to place an NDA on a finished product like the iPhone SDK (including the final version).
…
Apple then decided that it was a good idea to charge people for the privilege to develop for the iPhone: $99
…
These were two blatant mistakes, but, if you can believe it, Apple managed to alienate developers further still. A few thousand people put up with the NDA on the SDK, with the cost of the Standard Program, and with the lengthy and bureaucratic process it takes to access the only viable distribution channel, the iPhone App Store. Some of them spent months trying to create excellent, innovative applications for the iPhone, only to see their work rejected for no good reason other than that it competed with Apple’s own products (e.g. Podcaster) or was inconvenient for their business partner AT&T (e.g. NetShare).
…
I fail to see Apple’s usual business insight and only see blind greed, the kind that acts as a highly effective cautionary tale against developing for Apple’s platforms. This all comes at a time when Google is promoting a truly open platform, Android, which poses a few challenges due to the heterogeneous nature of the devices it will be deployed on, but is equally interesting from a technical standpoint. Google even went so far as to award ten million dollars in prize money through a contest that they held, to attract new developers and applications. Android is definitely welcoming new developers and it’s doing so free from glaring restrictions and limitations. I suspect that many will put up with Java, to get a cup of freedom.

This kind of thinking is particularly naive because it fails to consider why developers adopt platforms in the first place. Developers go where the users are. Users go where they can get the best user experience for the right price. Openness of the platform only helps if it improves the user experience, thus attracting more users and reinforcing the virtuous cycle.

Rory Blyth recently a very insightful which compared to "open" approach taken with the Windows Mobile developer ecosystem with the "closed" approach of Apple's iPhone ecosystem. In his post entitled iPhone vs. Windows Mobile - Apple vs. Microsoft - It's the Little Things Rory wrote

Here, from what I've learned, is how iPhone and Windows Mobile rate against these criteria.

---- Windows Mobile

My access to your money: If you have a WM device, you probably have money. Even with carrier "discounts" they're not cheap. If you know what to get, it's worth the moolah, and you'll take advantage of what your chosen device has to offer by downloading apps that make use of it. This is as easy as:

1. Trying to figure out where the big app stores are. There are a few, and they don't all support Windows Mobile, nor do they all have apps that will run on whatever version of WM you've got. It can be frustrating because Microsoft's practice of renaming products and slapping weird version numbers on things that are meaningless without context can easily leave you wondering what version of Windows Mobile you have (and wondering if what you've got is  the same as/compatible with PocketPC, WinCE, PPC, etc.).
2. If not big stores, then searching the small independents where devs post their stuff on sites that look like they were made with a beta release of the first version of FrontPage.
3. When you find an app, you go through whatever arbitrary transaction process the store/dev is using. This might mean creating an account with a site you'll never return to, handing your credit card info over to an independent whose trustworthiness is unknown, or even going through PayPal and then having to wait for the dev to check his email and manually respond with a serial number (or whatever).
4. Run the app on your desktop which will kick off ActiveSync 's install bits that install stuff on your PC in addition to the device.
5. After clicking "Yes" or "I think so" or "Sure" on a few dialog boxes that pop up on the desktop and on the device, a CAB file is opened on the device and a local installer runs. This can mean more dialog boxes, and it can also mean having to make choices about things you don't understand (many users aren't going to comprehend the impact/difference between installing to the device's memory or to an expansion card).
6. Run the app! Easy as 1-2-3-4-5-6!

…

App distribution options and ease of install for the customer: As you may have figured out from my "Easy as 1-2-3-4-5-6!" list above, finding, buying, and installing apps on WM devices has always been a pain in the ass. Going back to my first PocketPC (the first iPaq (the 3630)), I wondered why I needed ActiveSync just to install some stupid little app. ActiveSync makes sense if, say, I'm syncing something with the desktop like mail or calendar data, but it doesn't make sense if I'm installing Super Solitaire 5000 Deluxe Color Edition. Where do you sell your app? How do you get the word out? I haven't looked into it for a while because it frustrated me so much in the past. I'm going to take a look again, and, because I plan to target one specific platform for my app, I also plan to develop for others. In the case of Windows Mobile, I'm hoping Microsoft copies Apple's model.

---- Apple's iPhone

…

App distribution options and ease of install for the customer: Apple users have been bitching about using iTunes to install iPhone software. If they had any idea what it's like with other platforms, they'd shut it. While iTunes as an app store feels wrong and stupid and lame and stupid to me, at least iTunes is an app everybody has nowadays (aight - not everybody, but many, and that's good enough). Not that it matters much - with the introduction of Apple's App Store , you can browse apps on your phone, pay for, and install them without having to do some stupid syncing thing. You could be out at a bar where Jolene Blalock is hitting on you, and without having to run home to your iMac, you can buy, install, and run a crossword game before you've even had the chance to realize you've just made the biggest mistake in your life by ignoring her. And when you do realize it, and you see Jolene running off with another man, at least you'll have your crossword puzzles.

This is just one example of how a "closed" approach where a vendor supplies the entire end-to-end user experience provides a superior experience to an "open" approach where the vendor leaves it up to other developers to fill in the gaps. Apple's approach seems to be working well for developers some of whom are making hundreds of thousands of dollars a month thanks to how good of a job Apple has done in making it easy for users to find, purchase and install applications on their iPhones.

The key thing Apple has brought to the table is building a user experience that its customers love to use instead of one that they merely tolerate. Getting this right is way more important than the "openness" of the ecosystem. Customers and developers can put up with a closed ecosystem that limits choice as long as it improves the quality of the user experience. Where Google Android has to shine is in building a better user experience for the same price or a comparable experience at a lower price. Everything else is just noise.

Don't take my word for it, here's what the John Wang of HTC [Chief Marketing Officer of the company that is shipping the first Google Android phone] has to say about the topic in an article from Digitimes

Some believe the success of Android handsets will rely on their open source platform. However, this is not true since Linux-based handsets have already been on the market for a while, Wang argued.

The key element is innovation, said Wang, noting that the T-Mobile G1 is being rolled by combining Google's Internet services, HTC's proven capability in smartphone manufacturing, and T-Mobile's telecom network resources.

Apple is definitely ticking off developers but until another vendor shows up with a phone whose hardware and software provides a better experience for customers then it will continue to get the lions share of attention from top mobile developers.

I've been thinking a lot about platform adoption recently. I guess it is the combination of the upcoming Microsoft PDC and watching the various moves in the area of social networking platforms like OpenSocial and fbOpen. One thing that is abundantly clear is that the dynamics that drive platform adoption are amazingly consistent regardless of whether you are talking about operating system platforms like Windows' Win32 and *nix's POSIX, cloud computing platforms like Amazon's EC2 + EBS + S3 and Google App Engine, or even data access APIs like the Flickr API and Google's GData.

When a developer adopts a platform, there is a value exchange between the developer and the software vendor. The more value that is provided to developers by the platform vendor, the more developers are attracted to the platform. Although this seems self evident, where providers of platforms go astray is that they often don't understand the value developers actually want out of software platforms and instead operate from an if we build it they will come mentality. 

There are three main benefits adoption of one platform over another can offer a developer. These benefits are captured in the following "laws" of platform adoption  

1. Developers adopt a platform when it offers differentiation from competitors: In competitive software markets, building an application that stands out from the crowd is important. Platforms or technologies that enable developers to provide features that are unique to the application, even just temporarily, are thus valuable to developers in such markets. Typically, if these features are truly valuable to end users this leads to a "keeping up with the Joneses" effect where the majority applications in that space eventually adopt this platform. Recent examples of this include Flash video, AJAX and the Flickr API.

2. Developers adopt a platform when it reduces the cost of software development: Building software is a labor intensive, complicated and error prone process which often results in failure. This makes software development an expensive undertaking. Platforms which reduce the cost of development are thus very valuable to developers. Platforms like Java and the .NET Framework reduced the cost of software development compared to building applications using C and C++. In recent years, platforms based on dynamic languages such as Ruby On Rails and Django have become increasingly popular for building Web applications as they offer simpler development options compared to using "enterprise" platforms like Java.

3. Developers adopt a platform when it provides reach and/or better distribution: Every platform choice places a limit on which users the developer can reach. Building a Facebook application limits you to building applications for Facebook users, building an iPhone application limits you to people running Apple's phone and building an AJAX application limits you to users of modern browsers who don't have JavaScript disabled. In all of the aforementioned cases, the reach of the application platform is in the millions of users. Additionally, in all of the aforementioned cases there are alternative platforms that developers could choose that do not as many addressable end users and thus have less reach. 

   Distribution is another value add that modern platforms have begun to offer which augments the reach of the platform. The Facebook platform offers several viral distribution mechanisms for applications which enables applications to get noticed by end users and spread organically among users without explicit action by the developer. The Apple iPhone has the App Store which is the single, well-integrated entry point for users to discover and purchase applications for the device. Thus even though the iPhone may have users than other smart phone platforms, targeting the iPhone platform may still be more attractive than targeting other platforms due to its superior distribution channel for applications built on its platform. 

Sometimes developers adopt platforms due to external effects such as management mandate or peer pressure but even in these cases the underlying justification is usually one or more of the benefits above. These laws mean different things for the various participants in the software ecosystem.

• For software vendors, they clarify that delivering a software platform isn't just about delivering a technology or a set of APIs. The value proposition to developers with regards to the three laws of adoption must be clearly shown to get developers to accept the platform. Smart platform vendors should pick one or more of these axes as the value proposition of their platform and hammer it home. Good examples of these approaches include how Google has been hammering home the 'reach' benefit of adopting OpenSocial or how Erlang evangelists have been pitching it as a solution to the multicore crises given that building concurrent applications is still an expensive endeavor in today's popular programming languages. 

• For developers, they explain how to evaluate a platform and why some platforms get traction among their peers while others do not. So if you are a fan of the Common Lisp or the Pownce API, these laws of platform adoption explain why developers have flocked to other platforms in their stead.

Now Playing: Pink - So What

I've been reading about the Ning vs. WidgetLaboratory drama on TechCrunch. The meat of the conflict seems to be that widgets from WidgetLaboratory were so degrading the user experience of Ning that they had to be cut off. The relevant excerpts from the most recent TechCrunch story on the war of words are below

For those of you not closely following the drama between social network platform Ning and a popular widget provider called WidgetLaboratory, you can read the background here . On Friday Ning unceremoniously shut down their access to Ning, making all those widgets vanish.
...
In an email to WL on August 2 (more than three weeks ago), CEO Gina Bianchini wrote “Our only goal is to have you build your products in such a way that doesn’t slow down the networks running your products or takedown the Ning Platform with what you’re doing. Both of those would result in us needing to shutdown WidgetLaboratory products and that’s has never been our first choice of options. Hopefully, you know this after 8 months of working with us.”

Ignoring the he said, she said nature of the communication between both companies, there is a legitimate concern that 3rd party widgets included on the pages of a Web site can degrade the performance to the extent that the site becomes unusably slow. In fact, TechCrunch has had similar problems with 3rd party widgets as Mike Arrington has mentioned on his personal blog which led to him excluding the widgets from his site.

Typically, widgets are embedded in a site by including references to Javascript hosted on a 3rd party site in the page's HTML. This means rendering the page is dependent on how quickly the script files can be downloaded from the 3rd party site AND how long it takes for the script to execute especially since it may also fetch data from one or more servers as well. Thus a slow server or a badly written script can make every page that embeds the widget unbearably slow to render. Given that the ability to embed widgets is a key feature of social networking sites, it is important for such sites to figure out how to isolate their user experience from badly written widgets or widgets hosted on slow Web servers.

Below are some best practices that have emerged on how social networking sites can immunize themselves from the kinds of problems Ning has had with WidgetLaboratory

1. Host the Scripts Yourself: If you have a popular site, it is quite likely that you have more resources to handle lots of page views than the typical widget developer. Thus it makes sense to take away the dependency on externally hosted scripts by hosting the widgets yourself. Microsoft encourages developers to submit their gadgets to Windows Live Gallery if they want to build gadgets for my.live.com or Windows Live Spaces. For it's AJAX homepage service, Google does not require developers to submit gadgets to them for hosting but instead caches gadget data for hours at a time which means they are effectively hosting the gadgets themselves for the majority of the accesses by their users.

2. Keep External Dependencies off of Pages that Need to Render Quickly: In many cases, it isn't feasible to host all of the data and content related to widgets that are being shown on your site. In that case, you should ensure that the key scenarios on your Web site are insulated from the problems caused by slow or broken 3rd party widgets. For example, on Facebook viewing someone's profile is a key part of the user experience that is important to make sure happens as quickly and as smoothly as possible. For this reason, Facebook caches all 3rd party content that shows up on a user's profile and requires applications to call Profile.SetFBML to add content to the profile instead of providing a way to directly embed widgets on a user's profile.

3. Make It Clear Who Is to Blame if Things go Awry: One of the issues raised by Ning in their conflict with WidgetLaboratory is that user pages wouldn't render correctly or would show degraded performance due to WidgetLaboratory's widgets but Ning would get the support calls. This kind of user confusion is avoided if the user experience makes it clear when the failure of a page to render correctly is the fault of the external widget and when it is part of the hosting site. For example, Facebook Canvas Pages for applications make it clear that the user is using a 3rd party application and not part of the core Facebook experience. I've seen lots of user complain about the slowness of Scrabulous and Scrabble but never seen anyone who thought that Facebook was to blame and not the application developers.

Following some of these practices would have saved Ning and its users some of their current grief.

Now Playing: Ice Cube - Get Money, Spend Money, No Money

About a week ago, the Facebook Data team quietly released the Cassandra Project on Google Code. The Cassandra project has been described as a cross between Google's BigTable and Amazon's Dynamo storage systems. An overview of the project is available in the SIGMOD presentation on Cassandra available at SlideShare. A summary of the salient aspects of the project follows.

The problem Cassandra is aimed at solving is one that plagues social networking sites or any other service that has lots of relationships between users and their data. In such services, data often needs to be denormalized to prevent having to do lots of joins when performing queries. However this means the system needs to deal with the increased write traffic due to denormalization. At this point if you're using a relational database, you realize you're pretty much breaking every major rule of relational database design. Google tackled this problem by coming up with BigTable. Facebook has followed their lead by developing Cassandra which they admit is inspired by BigTable. 

The Cassandra data model is fairly straightforward. The entire system is a giant table with lots of rows. Each row is identified by a unique key. Each row has a column family, which can be thought of as the schema for the row. A column family can contain thousands of columns which are a tuple of {name, value, timestamp} and/or super columns which are a tuple of {name, column+} where column+ means one or more columns. This is very similar to the data model behind Google's BigTable.

As I mentioned earlier, denormalized data means you have to be able to handle a lot more writes than you would if storing data in a normalized relational database. Cassandra has several optimizations to make writes cheaper. When a write operation occurs, it doesn't immediately cause a write to the disk. Instead the record is updated in memory and the write operation is added to the commit log. Periodically the list of pending writes is processed and write operations are flushed to disk. As part of the flushing process the set of pending writes is analyzed and redundant writes eliminated. Additionally, the writes are sorted so that the disk is written to sequentially thus significantly improving seek time on the hard drive and reducing the impact of random writes to the system. How important is improving seek time when accessing data on a hard drive? It can make the difference between taking hours versus days to flush a hundred gigabytes of writes to a disk. Disk is the new tape.

Cassandra is described as "always writable" which means that a write operation always returns success even if it fails internally to the system. This is similar to the model exposed by Amazon's Dynamo which has an eventual consistency model.  From what I've read, it isn't clear how writes operations that occur during an internal failure are reconciled and exposed to users of the system. I'm sure someone with more knowledge can chime in in the comments.

At first glance, this is a very nice addition to the world of Open Source software by the Facebook team. Kudos.

Found via James Hamilton.

PS: Is it me or is this the second significant instance of Facebook Open Sourcing a key infrastructure component "inspired" by Google internals?

Now Playing: Ray J - Gifts

In the past year both Google and Facebook have released the remote procedure call (RPC) technologies that are used for communication between servers within their data centers as Open Source projects. 

Facebook Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. It supports the following programming languages; C++, Java, Python, PHP and Ruby.

Google Protocol Buffers allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. It supports the following programming languages; C++, Java and Python.

That’s interesting. Didn’t Steve Vinoski recently claim that RPC and it's descendants are "fundamentally flawed"? If so, why are Google and Facebook not only using RPC but proud enough of their usage of yet another distributed object RPC technology based on binary protocols that they are Open Sourcing them? Didn’t they get the memo that everyone is now on the REST + JSON/XML bandwagon (preferrably AtomPub)?

In truth, Google is on the REST + XML band wagon. Google has the Google Data APIs (GData) which is a consistent set of RESTful APIs for accessing data from Google's services based on the Atom Publishing Protocol aka RFC 5023. And even Facebook has a set of plain old XML over HTTP APIs (POX/HTTP) which they incorrectly refer to as the Facebook REST API.

So what is the story here?

It is all about coupling and how much control you have over the distributed end points. On the Web where you have little to no control over who talks to your servers or what technology they use, you want to utilize flexible technologies that make no assumptions about either end of the communication. This is where RESTful XML-based Web services shine. However when you have tight control over the service end points (e.g. if they are all your servers running in your data center) then you can use more optimized communications technologies that add a layer of tight coupling to your system. An example of the kind of tight coupling you have to live with is that  Facebook Thrift requires specific versions of g++ and Java if you plan to talk to it using code written in either language and you can’t talk to it from a service written in C#.

In general, the Web is about openness and loose coupling. Binary protocols that require specific programming languages and runtimes are the exact opposite of this. However inside your Web service where you control both ends of the pipe, you can optimize the interaction between your services and simplify development by going with a binary RPC based technology. More than likely different parts of your system are already doing this anyway (e.g. memcached uses a binary protocol to talk between cache instances, SQL Server uses TDS as the communications protocol between the database and it's clients, etc).

Always remember to use the right tool for the job. One size doesn’t fit all when it comes to technology decisions.

FURTHER READING

• Exposing a WCF Service With Multiple Bindings and Endpoints – Keith Elder describes how Windows Communication Foundation (WCF) supports multiple bindings that enable developers to expose their services in a variety of ways.  A developer can create a service once and then expose it to support net.tcp:// or http:// and various versions of http:// (Soap1.1, Soap1.2, WS*, JSON, etc).  This can be useful if a service crosses boundaries between the intranet and the Internet.

Now Playing: Pink - Family Portrait

Gnip is a newly launched startup that pitches itself as a service that aims to “make data portability suck less”. Mike Arrington describes the service in his post Gnip Launches To Ease The Strain On Web Services which is excerpted below

A close analogy is a blog ping server ( see our overview here ). Ping servers tell blog search engines like Technorati and Google Blog Search when a blog has been updated, so the search engines don’t have to constantly re-index sites just to see if new content has been posted. Instead, the blog tells the ping server when it updates, which tells the search engines to drop by and re-index. The creation of the first ping server, Weblogs.com, by Dave Winer resulted in orders of magnitude better efficiency for blog search engines.

The same thinking basically applies to Gnip. The idea is to gather simple information from social networks - just a username and the fact that they created new content (like writing a Twitter message, for example). Gnip then distributes that data to whoever wants it, and those downstream services can then access the core service’s API, with proper user authentication, and access the actual data (in our example, the actual Twitter message).

From a user’s perspective, the result is faster data updates across services and less downtime for services since their APIs won’t be hit as hard.

From my perspective, Gnip also shares some similarity to services like FeedBurner as well as blog ping servers. The original purpose of blog ping servers was to make it cheaper for services like Technorati and Feedster to index the blogosphere without having to invest in a Google-sized server farm and crawl the entire Web every couple of minutes. In addition, since blogs often have tiny readerships and are thus infrequently linked to, crawling alone was not enough to ensure that they find their way into the search index. It wasn’t about taking load off of the sites that were doing the pinging.

On the other hand, FeedBurner hosts a site’s RSS feed as a way to take load off of their servers and then provides analytics data so the site doesn’t miss out from losing the direct connection to its subscribers. This is more in line with the expectation that Gnip will take load off of a service’s API servers. However unlike FeedBurner, Gnip doesn’t actually store the user data from the social networking site. It simply stores a record that indicates that “user X on site Y made an update of type Z at time T”.  The thinking is that web sites will publish a notification to Gnip whenever their users perform an update. Below is a sample interaction between Digg and Gnip where Digg notifies Gnip that the users amy and john.doe have dugg two stories.

            ===> POST /publishers/digg/activity.xml Accept: application/xml
Content-Type: application/xml <activities> <activity at="2008-06-08T10:12:42Z"
uid="amy" type="dugg" guid="http://digg.com/odd_stuff/a_story"/> <activity at="2008-06-09T09:14:07Z"
uid="john.doe" type="dugg" guid="http://digg.com/odd_stuff/really_weird"/> </activities>
<--- 200 OK Content-Type: application/xml <result>Success</result> 

          

There are two modes in which "subscribers" can choose to interact with the data published to Gnip. The first is in a mode similar to how blog search engines interact with the changes.xml file on Weblogs.com and other blog ping servers. For example, services like Summize or TweetScan can ask Gnip for the last hour of changes on Twitter instead of whatever mechanism they are using today to crawl the site. Below is what a sample interaction to retrieve the most recent updates on Twitter from Gnip would look like

            ===>

GET /publishers/twitter/activity/current.xml

Accept: application/xml


<---

200 OK

Content-Type: application/xml


<activities>

<activity at="2008-06-08T10:12:07Z" uid="john.doe" type="tweet" guid="http://twitter.com/john.doe/statuses/42"/>

<activity at="2008-06-08T10:12:42Z" uid="amy" type="tweet" guid="http://twitter.com/amy/statuses/52"/>

</activities> 
          

The main problem with this approach is the same one that affects blog ping servers. If the rate of updates is more than the ping server can handle then it may begin to fall behind or lose updates completely. Services that don’t want to risk their content not being crawled are best off providing their own update stream that applications can poll periodically. That’s why the folks at Six Apart came up with the Six Apart Update Stream for LiveJournal, TypePad and Vox weblogs.

The second mode is one that has gotten Twitter fans like Dave Winer raving about Gnip being the solution to Twitter’s scaling problems. In this mode, an application creates a collection of one or more usernames they are interested in. Below is what a collection document created by the Twadget application to indicate that it is interested in my Twitter updates might look like.

<collection name="twadget-carnage4life">
     <uid name="carnage4life" publisher.name="twitter"/>
</collection>

Then instead of polling Twitter every 5 minutes for updates it polls Gnip every 5 minutes for updates and only talks to Twitter’s servers when Gnip indicates that I’ve made an update since the last time the application polled Gnip. The interaction between Twadget and Gnip would then be as follows

            ===>

GET /collections/twadget-carnage4life/activity/current.xml

Accept: application/xml

<---

200 OK

Content-Type: application/xml


<activities>

<activity at="2008-06-08T10:12:07Z" uid="carnage4life" type="tweet" guid="http://twitter.com/Carnage4Life/statuses/850726804"/>
            
            
            

             </activities> 
          

Of course, this makes me wonder why one would think that it is feasible for Gnip to build a system that can handle the API polling traffic of every microblogging and social networking site out there but it is infeasible for Twitter to figure out how to handle the polling traffic for their own service. Talk about lowered expectations.

So what do I think of Gnip? I think the ping server mode may be of some interest for services that think it is cheaper to have code that pings Gnip after every user update instead building out an update stream service. However since a lot of sites already have some equivalent of the public timeline it isn’t clear that there is a huge need for a ping service. Crawlers can just hit the public timeline which I assume is what services like Summize and TweetScan do to keep their indexes of tweets up to date.

As for using Gnip as a mechanism for reducing the load API clients put on a microblogging or similar service? Gnip is totally useless for that in it’s current incarnation. API clients aren’t interested in updates made by single user. They are interested in all the updates made by all the people the user is following. So for Twadget to use Gnip to lighten the load it causes on Twitter’s servers on my behalf, it has to build a collection of all the people I am following in Gnip and then keep that list of users in sync with whatever that list is on Twitter. But if it has to constantly poll Twitter for my friend list, isn’t it still putting the same amount of load on Twitter? I guess this could be fixed by having Twitter publish follower/following lists to Gnip but that introduces all sorts of interesting technical and privacy issues. But that doesn’t matter since the folks at Gnip brag about only keeping 60 minutes of worth of updates as the “secret sauce” to their scalability. This means if I shut my Twitter client hasn’t polled Gnip in a 60 minute window (maybe my laptop is closed) then it doesn’t matter anyway and it has to poll Twitter.  I suspect someone didn’t finish doing their homework before rushing to “launch” Gnip.

PS: One thing that is confusing to me is why all communication between applications and Gnip needs to be over SSL. The only thing I can see it adding is making it more expensive for Gnip run their service. I can’t think of any reason why the interactions described above need to be over a secure channel.

Now Playing: Lil Wayne - Hustler Musik

Late last week, the folks on the Google Data APIs blog announced that Google will now be supporting OAuth as the delegated authentication mechanism for all Google Data APIs. This move is meant to encourage the various online services that provide APIs that access a user’s data in the “cloud” to stop reinventing the wheel when it comes to delegated authentication and standardize on a single approach.

Every well-designed Web API that provides access to a customer’s data in the cloud utilizes a delegated authentication mechanism which allows users to grant 3rd party applications access to their data without having to give the application their username and password. There is a good analogy for this practice in the OAuth: Introduction page which is excerpted below

What is it For?

Many luxury cars today come with a valet key. It is a special key you give the parking attendant and unlike your regular key, will not allow the car to drive more than a mile or two. Some valet keys will not open the trunk, while others will block access to your onboard cell phone address book. Regardless of what restrictions the valet key imposes, the idea is very clever. You give someone limited access to your car with a special key, while using your regular key to unlock everything.

Everyday new website offer services which tie together functionality from other sites. A photo lab printing your online photos, a social network using your address book to look for friends, and APIs to build your own desktop application version of a popular site. These are all great services – what is not so great about some of the implementations available today is their request for your username and password to the other site. When you agree to share your secret credentials, not only you expose your password to someone else (yes, that same password you also use for online banking), you also give them full access to do as they wish. They can do anything they wanted – even change your password and lock you out.

This is what OAuth does, it allows the you the User to grant access to your private resources on one site (which is called the Service Provider), to another site (called Consumer, not to be confused with you, the User). While OpenID is all about using a single identity to sign into many sites, OAuth is about giving access to your stuff without sharing your identity at all (or its secret parts).

So every service provider invented their own protocol to do this, all of which are different but have the same basic components. Today we have Google AuthSub, Yahoo! BBAuth, Windows Live DelAuth, AOL OpenAuth, the Flickr Authentication API, the Facebook Authentication API and others. All different, proprietary solutions to the same problem.

This ends up being problematic for developers because if you want to build an application that talks to multiple services you not only have to deal with the different APIs provided by these services but also the different authorization/authentication models they utilize as well. In a world where “social aggregation” is becoming more commonplace with services like Plaxo Pulse & FriendFeed and more applications are trying to bridge the desktop/cloud divide like OutSync and RSS Bandit, it sucks that these applications have to rewrite the same type of code over and over again to deal with the basic task of getting permission to access a user’s data. Standardizing on OAuth is meant to fix that. A number of startups like Digg & Twitter as well as major players like Yahoo and Google have promised to support it, so this should make the lives of developers easier.

Of course, we still have work to do as an industry when it comes to the constant wheel reinvention in the area of Web APIs. Chris Messina points to another place where every major service provider has invented a different proprietary protocol for doing the same task in his post Inventing contact schemas for fun and profit! (Ugh) where he writes

And then there were three
...
Today, Yahoo! announced the public availability of their own Address Book API .

However, I have to lament yet more needless reinvention of contact schema. Why is this a problem? Well, as I pointed out about Facebook’s approach to developing their own platform methods and formats, having to write and debug against yet another contact schema makes the “tax” of adding support for contact syncing and export increasingly onerous for sites and web services that want to better serve their customers by letting them host and maintain their address book elsewhere.

This isn’t just a problem that I have with Yahoo!. It’s something that I encountered last November with the SREG and proposed Attribute Exchange profile definition . And yet again when Google announced their Contacts API . And then again when Microsoft released theirs ! Over and over again we’re seeing better ways of fighting the password anti-pattern flow of inviting friends to new social services, but having to implement support for countless contact schemas. What we need is one common contacts interchange format and I strongly suggest that it inherit from vcard with allowances or extension points for contemporary trends in social networking profile data.

I’ve gone ahead and whipped up a comparison matrix between the primary contact schemas to demonstrate the mess we’re in.

Kudos to the folks at Google for trying to force the issue when it comes to standardizing on a delegated authentication protocol for use on the Web. However there are still lots of places across the industry where we speak different protocols and thus incur a needless burden on developers when a single language might do. It would be nice to see some of this unnecessary redundancy eliminated in the future.

Now Playing: G-Unit - I Like The Way She Do It

Matt Asay of C|Net has an article entitled Facebook adopts the CPAL poison pill where he writes

Instead, by choosing CPAL , Facebook has ensured that it can be open source without anyone actually using its source. Was that the intent?

As OStatic explains , CPAL requires display of an attribution notice on derivative works. This practice, which effectively requires downstream code to carry the original developer(s)' logo, came to be known as " badgeware ." It was approved by the OSI but continues to be viewed with suspicion within the open-source community.

I've written before about how most open-source licenses don't apply themselves well to the networked economy . Only the OSL, AGPL, and CPAL contemplate web-based services. It's not surprising that Facebook opted for one of these licenses, but I am surprised it chose the one least likely to lead to developers actually modifying the Facebook platform.

If the point was to protect the Facebook platform from competition (i.e., derivative works), Facebook chose a good license. If it was to encourage development, it chose the wrong license.

But if the purpose was to prevent modifications of the platform, why bother open sourcing it at all?

I've seen more than one person repeat the sentiment in the above article which leaves me completely perplexed. With fbOpen Facebook has allowed anyone who is interested to run Facebook applications and participate in what is currently the most popular & vibrant social network widget ecosystem in the world.

I can think of lots of good reasons for not wanting to adopt fbOpen. Maybe the code is in PHP and you are a Ruby On Rails shop. Or maybe it conflicts with your company's grand strategy of painting Facebook as the devil and you the heroes of openness (*cough* Google *cough*). However I can't see how requiring that you mention somewhere on your site that your social network's widget platform is powered by the Facebook developer platform is some sort of onerous POISON PILL which prevents you from using it. In the old days, companies used to charge you for the right to say your application is compatible with theirs, heck, Apple still does. So it seems pretty wacky for someone to call Facebook out for letting people use their code and encouraging them to use the Facebook brand in describing their product. Shoot!

The premise of the entire article is pretty ridiculous, it's like calling the BSD License a poison pill license because of the advertising clause. This isn't to say there aren't real issues with an advertising clause as pointed out in the GNU foundation's article The BSD License Problem. However as far as I'm aware,  adopters of fbOpen don't have to worry about being obligated to display dozens powered by X messages because every bit of code they depend on requires that it is similarly advertised. So that argument is moot in this case.

Crazy article but I've come to expect that from Matt Asay's writing.

Now Playing: Eminem & D12 - Sh*t On You

A few months ago Michael Mace, former Chief Competitive Officer and VP of Product Planning at Palm, wrote an insightful and perceptive eulogy for mobile application platforms entitled Mobile applications, RIP where he wrote

Back in 1999 when I joined Palm, it seemed we had the whole mobile ecosystem nailed. The market was literally exploding, with the installed base of devices doubling every year, and an incredible range of creative and useful software popping up all over. In a 22-month period, the number of registered Palm developers increased from 3,000 to over 130,000. The PalmSource conference was swamped, with people spilling out into the halls, and David Pogue took center stage at the close of the conference to tell us how brilliant we all were.

It felt like we were at the leading edge of a revolution, but in hindsight it was more like the high water mark of a flash flood.
...

Two problems have caused a decline the mobile apps business over the last few years. First, the business has become tougher technologically. Second, marketing and sales have also become harder.

From the technical perspective, there are a couple of big issues. One is the proliferation of operating systems. Back in the late 1990s there were two platforms we had to worry about, Pocket PC and Palm OS. Symbian was there too, but it was in Europe and few people here were paying attention. Now there are at least ten platforms. Microsoft alone has several -- two versions of Windows Mobile, Tablet PC, and so on. [Elia didn't mention it, but the fragmentation of Java makes this situation even worse.]

I call it three million platforms with a hundred users each (link).

 

So what does this have to do with social networking sites? The first excerpt from the post where it talks about 130,000 registered developers for the Palm OS sounds a lot like the original hype around the Facebook platform with headlines screaming Facebook Platform Attracts 1000 Developers a Day.

The second excerpt talks about the time there became two big mobile platforms, analogous to the appearance of Google's OpenSocial on the scene as a competing platform used by a consortium of Facebook's competitors. This means that widget developers like Slide and RockYou have to target one set of APIs when building widgets for MySpace, LinkedIn, & Orkut and another completely different set of APIs when building widgets for Facebook & Bebo. Things will likely only get worse. One reason for this is that despite API standardization, all of these sites do not have the same features. Facebook has a Web-based IM, Bebo does not. Orkut has video uploads, LinkedIn does not. All of these differences eventually creep into such APIs as "vendor extensions". The fact that both Google and Facebook are also shipping Open Source implementations of their platforms (Shindig and fbOpen) makes it even more likely that the social networking sites will find ways to extend these platforms to suit their needs.

Finally, there's the show-me-the-money problem. It still isn't clear how one makes money out of building on these social networking platforms. Although companies like Photobucket and Slide have gotten a quarter of a billion and half a billion dollar valuations these have all been typical "Web 2.0" zero profit valuations. This implies that platform developers don't really make money but instead are simply trying to gather a lot of eyeballs then flip to some big company with lots of cash and no ideas. Basically it's a VC funded lottery system. This doesn't sound like the basis of successful and long lived platform such as what we've seen with Microsoft's Windows and Office platforms or Google's Search and Adwords ecosystem. In these platforms there are actually ways for companies to make money by adding value to the ecosystem, this seems more sustainable in the long run than what we have today in the various social networking widget platforms.

It will be interesting to see if the history repeats itself in this instance.

Now Playing: Rick Ross - Luxury Tax (featuring Lil Wayne, Young Jeezy and Trick Daddy)