There’s nothing like a successful company with a near monopoly to force the software industry to come up with standards. Or in this case, as in many others, force it’s competitors to band together and call what they are doing the standard because more than one vendor supports it.

From TechCrunch’s article Details Revealed: Google OpenSocial(To Launch Thursday we learn

Google wants to create an easy way for developers to create an application that works on all social networks. And if they pull it off, they’ll be in the center, controlling the network.

What They’re Launching

OpenSocial is a set of three common APIs, defined by Google with input from partners, that allow developers to access core functions and information at social networks:

  • Profile Information (user data)
  • Friends Information (social graph)
  • Activities (things that happen, News Feed type stuff)

Hosts agree to accept the API calls and return appropriate data. Google won’t try to provide universal API coverage for special use cases, instead focusing on the most common uses. Specialized functions/data can be accessed from the hosts directly via their own APIs.

Unlike Facebook, OpenSocial does not have its own markup language (Facebook requires use of FBML for security reasons, but it also makes code unusable outside of Facebook). Instead, developers use normal javascript and html (and can embed Flash elements). The benefit of the Google approach is that developers can use much of their existing front end code and simply tailor it slightly for OpenSocial, so creating applications is even easier than on Facebook.

Similar details are available from folks like Om Malik and Marc Andreesen

This is a brilliant move. I’ve blogged on multiple occassions that the disparate widget platforms in social networking sites is a burden for widget developers and will lead to a “winner takes all” situation because no one wants to support umpteen different platforms. If enough momentum gains around OpenSocial, then three things will happen

  • Widget developers will start to favor coding to OpenSocial because it supports multiple sites as well as targeting the Facebook platform  
  • Eventually Facebook platform developers will start asking Zuckerburg and company to support OpenSocial so they only need to worry about one code base (kinda, it won’t be that easy)  
  • Other companies with proprietary widget platforms or plans to create one will bow down to the tide and adopt OpenSocial

Of course, this requires a popular social networking site with a wide audience (e.g. MySpace) to adopt the platform before we see this kind of traction.

However this is the only thing Google could have done that makes any sense. Building a clone of the Facebook platform like some social networking sites planned would have been dumb because that would be the tail wagging the dog. Similarly building a competing proprietary platform would also have been dumb due to the winner takes all problem I mentioned earlier.

This is the only move that has a chance of actually giving their anti-Facebook platform a chance of being successful.

I wonder how my coworkers in Windows Live are going to take this news?

Now playing: 50 Cent - All Of Me (Feat. Mary J Blige) (Prod by Jake One)


Web companies are primarily media companies. Their job is to assemble audiences and then sell ads to their captive audiences. This is what the newspaper, television and magazine industries did before them but for some reason Web companies like Google love to claim that they aren’t media companies.  The irony is that Google stumbled on the most lucrative media property of all time (Web search results) as well as the most lucrative way of monetizing it (auction based keyword sales).

Web 1.0 was about gathering a bunch of eyeballs and then entertaining them with a bunch of content you created. Web 2.0 is all about gathering a bunch of eyeballs then having them entertain themselves with content they’ve created (also known as UGC). The most important thing hasn’t changed, it’s about gathering eyeballs.

This takes us to social networking sites like MySpace and Facebook which have taken the act of gathering eyeballs and then having your users entertain themselves with the help of random developers and elevated it to a fine art.  Things get interesting when you realize that a lot of these “random developers” have built compelling experiences and have gathered a significant audience in their own right on the backs of these social networking sites.

In a blog post entitled Scrabulous Jason Antonelli writes

I've become addicted to Scrabulous on Facebook…I'm on my 3rd game of Scrabble with my mom (yes I won two games so far) - and it usually starts with a phone call where we play for a bit and chat about various mom/son things.  My mom lives 3000+ miles away -- she's in Wilmington, NC and I'm in Seattle.  After a while when one of us needs to go do something else - we'll go ahead and play it throughout the day - maybe each making a 1-2 moves a day.

This is so much fun (though I wonder how much it would be if I didn't have a 100% win rate) and it clearly is bringing social networks into the mainstream.  The fact that this game is hosted on the Facebook platform (although it does look like it has a standalone site as well) gave two major wins to Facebook.  First I actually convinced by mom (I'm 33 so you can only guess her age!) to signup for Facebook - the first person I got to sign up to any social networking application even including the one I've worked on.  And second, both my mom and I log in to Facebook multiple times a day.

I’m similarly hooked on Scrabulous on Facebook and estimate that 75% of my page views to * are actually to the Scrabulous application page. I’m definitely see more ads hosted within Scrabulous than I see those hosted within Facebook*. Which leads me to wonder especially when I notice that Scrabulous now sports an ads by Google logo. I specifically wondered about the reach of the most popular widget makers and was surprised when I took a look at the About Page which informed me that  

Slide is the largest personal media network in the world, reaching more than 134 million unique global viewers each month and 30 percent of the U.S. Internet audience. We help people express themselves and tell stories through personalized photos and videos created on and viewed anywhere on the web or desktop.

Slide widgets — including Slideshows, Guestbooks, SkinFlix and FunPix — are popular on top social networking and blog platforms, including MySpace, Facebook, Bebo, Hi5, Friendster, Tagged, Piczo and Blogger. Slide is also the leading application developer on Facebook with more than 63 million applications installed, including SuperPoke and Top Friends, the most active application by more than four times that of any other 3rd party developer.

Whoa. I suspect that once MySpace ships their platform which relaxes the rules on advertising within widgets, companies like Slide will start to look like very interesting acquisition targets. Or maybe there’s enough money to be made generating the majority of the page views on the most popular social networking sites in the world, that there’ll be no incentive to cash out. Interesting times ahead.

* I do think I’ve clicked more on Facebook hosted ads than those hosted within Scrabulous so that may not be a useful metric. Smile

Now playing: Fabolous - This Is My Party


Dave Winer has an interesting post where he asks Facebook app or Firefox plug-in? 

Which is a more interesting platform -- Facebook or Firefox?

This was a topic of conversation at the Web 2.0 Summit last week in SF, not on stage, but in a LobbyConversation between myself and venture capitalist Bijan Sabet.

Bijan Sabet: "I like that Firefox developers don't have to live in a world where they lie awake at night worried that the platform company is going to make life hard for them."

You can guess my answer to the question so I won’t bother answering. I will point out that I doubt that the developers of Flock would agree with the statement that Firefox doesn’t do anything to screw developers on their platform. 

I also question anyone who claims that there are more opportunities to make money off of Firefox plugins than there are selling ads in your Facebook application.

Now playing: 2Pac - Fight Music (feat. Xzibit)


Categories: Platforms

October 26, 2007
@ 04:00 AM

Via Greg Linden I found the the summary of a recent survey 1,001 US adults conducted by Kelton Research. The summary was posted on Search Engine Land in an article entitled Report: 7 Out Of 10 Americans Experience 'Search Engine Fatigue' which states

The report discusses user frustration with clutter and the content of search results:

When asked to name their #1 complaint about the process, 25 percent cited a deluge of results, 24 percent cited a predominance of commercial (paid) listings, 18.8 percent blamed the search engine’s inability to understand their keywords (forcing them to try again), and 18.6 percent were most frustrated by disorganized/random results.

There was also a desire among many users that search engines be able to "read their minds":

Kelton asked survey respondents whether they wished that search engines like Google could, in effect, read their minds, delivering the results they were actually looking for. . . That capability is something that 78 percent of all survey-takers “wished” for, including 86.2 percent of 18-34 year-olds and 85 percent of those under 18.

That sounds like an argument for search personalization.

Search personalization is just one of the many ways to deal with end user frustrations with search results. For example, no amount of search personalization will be as effective as statistical analysis to discover and automatically fix mispellings in search queries (e.g. all the misspellings of britney spears Google has to deal with). Without spelling correction, a user will leave frustrated because they get few or no results when in truth they typed a misspelling into the search box.

The folks on the Live Search team have a first in a series of blog posts about how they tackle the problem of determining a user’s intent from their search queries entitled “Do what I mean, not what I say!” [Part 1 of 2]. It contains the following excerpt

We've been working on returning the very best search results for your intent, not just for the particular search terms that you happen to have chosen as a vehicle to transmit that intent.  There's an important difference there and it's been our focus for quite some time.

AutoSpell Correction

The first example of this is our new AutoSpell feature.

If we are absolutely, completely, totally, "no doubt about it" confident you misspelled one of your search terms, we automatically deliver a page that includes spell-corrected results, rather than a page of misspelled results accompanied by a "Did you mean _______?" link at the top.

For example, there's this pizza place near Microsoft called Pagliacci Pizza that is fantastic.  The problem is that I can never remember the correct spelling of the place.  My misspelled attempts are usually something along the lines of Pagliaci Pizza, Pagliaccis Pizza, or Paggliacci Pizza...


Another improvement in the "Do what I mean, not what I say" category is stemming.  Stemming means matching on the "stem" (or root) of the word rather than the exact word.

For example, users told us that the search half price book Redmond returned horrible results.  Searching for half price books Redmond produced much better results.   In our new release of Live Search, searches for half price book Redmond automatically include results with books in them as well.

Our team knew that tackling stemming would give us the improvements we needed for searches like these.  But we had to be careful, because you can't just stem all the time-you have to be smart about it.  An example of this is the word cable.  When you search for "cable," you could be looking for information on cable TV providers.  When you search for "cables", you could be looking for power, telephone, or network cables. 

These are just two techniques beyond just personalization which gives users better satisfaction with their search results and the impression that the search engine is “reading their mind”.

The main problem I have with personalization is that you need to give the search engine a private, personal information amount of information before it has tangible effects on search results. In the recent past, Marissa Mayer has pointed out that user studies have shown that location is the only significant factor which impacts perception of the relevance of search results when it comes to personalization. I suspect that true personalization will come from doing things like analysing my profile or my social graph friends list instead of the approach popularized by Google Personalized Search where previous search queries are analyzed.

Of course, we live in an era when 10% of the Internet population doesn’t see anything wrong with brain implants to connect them to the Web so maybe I’m being paranoid when I worry that the next major leap in search engine relevance will only occur after we allow search engines to spy on us.

Now playing: Busta Rhymes - I Know What You Want (feat. Mariah Carey)


Categories: Windows Live

October 26, 2007
@ 04:00 AM

Before we got married, Jenna and I talked about pets and both agreed we didn’t want any. We didn’t want some animal that would urinate on the carpet, need to be taken outside for walks where I’d have to pick up it’s excrement, and that would create an additional inconvenience every time we decided to go out of town.

I’m sure you know where this is going…

We pick him up next week. Since I’m so against the idea, I get to name him. So far I’m torn between “Socks” and “Buster” both of which my wife thinks are horrible names for a dog. Any suggestions?

Now playing: Snoop Doggy Dogg - Doggy Dogg World


Categories: Personal

The folks at LiveSide have a blog post entitled Windows Live Spaces at a crossroads: will the US catch up to the world? which contains some interesting charts from ComScore. Specifically they call out the difference in the worldwide reach of various social networking services versus the North American reach. The relevant excerpt from the post is

As expected, MySpace is in a runaway lead, Facebook is coming on strong, Blogger is hanging in there, and Spaces pretty much brings up the rear.  If you read the blogs and follow Techmeme, TechCrunch, and Scoble, these numbers aren't anything surprising.

But take a look at the Worldwide numbers, and a somewhat different story emerges.

Social Networking Sites - Worldwide: Unique visitors per month (000)


While Facebook is growing steadily worldwide, here the numbers tell a far different story.  Windows Live Spaces is battling it out with Blogger and MySpace for the top spot.  Just for reference, we can see that the Worldwide usage of Social Networking sites is growing steadily:

You might quibble with the title of this blog post but it is hard to argue that Blogger is a social networking site by any definition of the term. When it comes to reach, no social networking site impacts as many users as Windows Live Spaces.

Of course, unique users aren’t the only metric Web sites are judged against and I’m sure there are many out there who will be quick to point out other charts that show our user engagement is lower than average which is a fair point. Personally, I suspect that the inclusion of the improved What’s New page will increase user engagement in a measurable way. It might just be me but I find myself visiting my What’s New page several times a day, in fact more often than I visit my Facebook news feed.

According to Facebook, the addition of the News Feed increased their page views by 70% in the first few months. I wonder if we’ll see a similar jump in the ComScore charts for Windows Live Spaces in a few months or whether FB’s results were an abberration. Only time will tell.

Now playing: Raekwon - Guillotine (Swordz) (feat. Ghostface Killah, Inspectah Deck & GZA/Genius)


Categories: Social Software | Windows Live

Today I saw a pretty useless product announced on TechCrunch called FriendCSV which extracts your Facebook friends into a CSV file sans their PII (i.e. no email address, no IM screen names, no telephone numbers, no street address). Amusingly enough, the authors of the application brag about using the Facebook platform as it was designed to be used as if they have figured out some major exploit. Smile

However it did remind me of a pretty cool desktop application that I found via Bubba called OutSync. It synchronizes the photos of your contacts in Outlook with those of any matching people found on your friends list in Facebook. If you have a Windows Mobile smartphone that is synchronized with your Exchange server like I do, it means that you get all your friends photos on your phone which is pretty sweet especially when you get a call from that person. Screenshots below

OutSync access request screen

There’s that user-centric authentication model where the user grants the application access without giving the application their username/password. Again, I’m glad OAuth is standardizing this for the Web.

When I think of the Facebook platform embracing the “Web as a Platform” I want to see more applications like this enabled. Instead of only utilizing my Facebook social graph in the Facebook Marketplace or the Facebook application, why can’t it be utilized on eBay or Craigslist? I want all the applications I use to be able to utilize my social graph to make themselves better without having to be widgets on a particular social networking site before they can take advantage of this knowledge. 

Back in 2004, I wrote

Basically, we've gotten rid of one of major complaints about online services; maintaining to many separate lists of people you know. One of the benefits of this is that you can utilize this master contact list across a number of scenarios outside of just one local application like an email application or an IM client. For example, in MSN Spaces we allow users to use their MSN Messenger allow list (people you've granted permission to contact you via IM) as a access control list for who can view your Space (blog, photo album, etc). There are a lot more interesting things you can do once the applications you use can tell "here are the people I know, these are the ones I trust, etc". We may not get there as an industry anytime soon but MSN users will be reaping the benefits of this integration in more and more ways as time progresses.

It’s unfortunate that almost 3 years later we haven’t made much progress on this across the industry although it looks like the Facebook platform has gotten people finally thinking about unified social graphs. Better late than never, I guess. 

The one mistake I’ve been making in my thinking has been narrowly defining the applications that should have access to your Windows Live social graph as Microsoft or Windows Live applications. My thinking has since evolved as has that of lots of folks in the B0rg cube. It will be an interesting couple of months with regards to social graph APIs especially with Google’s November 5th announcement coming up.

PS: Anyone else noticed that the installer for OutSync is hosted on SkyDrive?

PPS: I got a phone call from Rob Dolin earlier this evening and the pic from his Facebook profile showed up my phone. Sweet!

Now playing: Soulja Boy - Crank That (Soulja Boy)


Categories: Platforms | Social Software

For a while, I’ve been jealous of the Flickr API Explorer and the Facebook API Test console. So I started building one for the Windows Live Contacts API last week only to find out that there was already one out there which hadn’t been sufficiently publicized.

If you’re a developer interested in seeing what the Windows Live Contacts API offers and already have a Windows Live ID (formerly Passport account) then mosey on down to the the Live Data Interactive SDK page. Once you get there, click on the link that says Click to Request Permission , scroll down and hit [Give Access] to give the page access to your Windows Live address book. Once redirected, click “Work with Contacts” and then you can create, retrieve, update and delete people from your Hotmail and Messenger contact lists using a simple RESTful protocol with direct access to the XML responses. 

Below is a screenshot of me retrieving my Messenger contact using the interactive SDK.

Now playing: Baby Bash - Cyclone (feat. T-Pain)


Categories: Windows Live | XML Web Services

I recently read two blog posts on Microsoft's VisitMix site recently which show how conflicted large Web players can be about embracing the fact that The Web is the Platform. The first is a blog post by Scott Barnes entitled Rich Interactive Applications which contains the following excerpt

When you think of RIA what is it your mind casts an image to first?
RIA isn't about attention/eyeballs, it's supposed to be focused on empowering end users of a defined type, to carry out mundane task through an enriching user experience. User Experience is the key, in that a true RIA solution has the power to abstract complexity through aggregation or 360 degree view(s) of content without altering context.
That is Rich Interactive Application (RIA) shifting the paradigm. It had nothing to do with the Internet, suffice to say it's housed within an agent which is connected to the Internet - or - Intranet.

This blog post was the first time I've seen the term RIA defined as Rich Interactive Application instead of Rich Internet Application. Digging a little, it becomes obvious that Scott Barnes's post is just one of many in an ongoing flame war war of words between developer evangelists at Microsoft and developer evangelists at Adobe.

Redefining a term in such a way that it becomes all-inclusive is a recipe for devaluing the term [which might be Scott's purpose]. This is the lesson from the all-inclusive definitions that started to swirl around industry terms like Service Oriented Architecture and Web 2.0. More importantly, the problem with using the term "Rich Interactive Application" to define what developers commonly describe as RIAs is that it completely misses the point. Developers and end users are not excited about the ability to build and use rich interactive applications, they are excited about being able to build and use rich interactive applications on the Web. They've had the former for as long as desktop computers have existed, the latter is what is currently jazzing people up (e.g. all the hype around AJAX, Flickr, YouTube, the Facebook platform, etc).

Don't fight the Web. People don't get excited about "interactive" desktop applications. When was the last time your best friend, mom, daughter, sister, co-worker, etc told you about some cool desktop app they just found or use regularly? How does that compare that to the amount of times they've told about cool Web sites they found or use regularly?

Think about that for a second, Mr. Rich Interactive Application. Embrace the Web or you will be left behind.

Onto Joshua Allen's post entitled Web is THE Platform? SRSLY? which states

Erick Schonfeld at TechCrunch reports on Google's presentation today at Web 2.0 Conference.  Jeff Huber of Google, trying to slam Facebook and MySpace, said "A lot that you have heard here is about platforms and who is going to win. That is Paleolithic thinking. The Web has already won. The web is the Platform. So let’s go build the programmable Web."

I was rather surprised, because I heard that same line just two days ago, from Dare Obasanjo.  Jeff apparently reads Dare's blog, and was in a hurry to prepare his speech.
When I hear someone talk about the web as a platform, I have a pretty clear picture:

  • Utilizes open standards, preferably mature specifications and preferably from W3C
  • Utilizes web client runtime that has massive deployment; depends only on functionality that can be found in the majority of browsers
  • Runs the same no matter who is hosting the code

This is non-negotiable!  When any normal person writes "for the web", this is what she means! 

Joshua goes on to cite Google for hypocrisy because it's widget platform is every bit as proprietary as those of MySpace and Facebook, and Google's doesn't use any of the ad-hoc standards for exposing social graph data in a shareable way (FOAF, XFN, etc).

Although all the things Joshua lists are important, they aren't what I was really harping on when I wrote the post referenced by Joshua. The problem with the Facebook platform is that although you can use it to build Web applications, they are not on the Web. What do I mean by being on the Web? Here's a sampling of writings from across the Web that does a better of job of explaining this than I ever could

Tim Berners-Lee

When I invented the Web, I didn't have to ask anyone's permission. Now, hundreds of millions of people are using it freely.

Jason Kottke

Faced with competition from this open web, AOL lost...running a closed service with custom content and interfaces was no match for the wild frontier of the web. Maybe if they'd done some things differently, they would have fared better, but they still would have lost. In competitive markets, open and messy trumps closed and controlled in the long run.

Anil Dash

It's not true to say that Facebook is the new AOL, and it's oversimplification to say that Facebook's API is the new Blackbird, or the new Rainman. But Facebook is part of the web. Think of the web, of the Internet itself, as water. Proprietary platforms based on the web are ice cubes. They can, for a time, suspend themselves above the web at large. But over time, they only ever melt into the water.


Categories: Platforms

I’ve been reading recently that a number of social networking sites are rushing to launch [or re-launch] a widgets platform given the success of the Facebook platform. There have been announcements about a MySpace platform which claim that

  • it will essentially be a set of APIs and a new markup language that will allow third party developers to create applications that run within MySpace. Developers will be able to include Flash applets, iFrame elements and Javascript snippets in their applications, and access most of the core MySpace resources (profile information, friend list, activity history, etc.). Unlike existing widgets on MySpace, developers will be able to access deep profile and other information about users and bake it into the applications.
  • Advertising can be included on the application pages (called control pages) and developers will keep 100% of the revenue. Ads may not be placed within widgets that appear on MySpace pages, however.
  • There have been similar announcements from LinkedIn and Google. The problem is that every one of these widget platforms being proposed by the various social networking sites are incompatible. This means that Web developers has to build a separate application for each of these sites using proprietary technologies (e.g. FBML) and proprietary APIs (e.g. FQL). Since very few Web developers or Web companies will be able to support significant applications on every one of these platforms, the question then becomes “Which platform should you bet on?” and “How do you make the decision to bet on a platform.”

    Right now the gold standard in widget platforms for social networking sites is the Facebook platform. There are several reasons for this and competitors planning to build similar platforms need to meet the following criteria.

    1. Monetization: Facebook encourages developers to monetize their widgets by placing ads in their widgets. Although Facebook has not actively helped developers by providing an ad platform, there is now a healthy marketplace of Facebook ad networks that developers can choose from. It has even been rumored that Google will be getting in the Facebook ad provider game. +1 to Facebook.

    2. Distribution and Reach: A big problem you’ll face when you’ve built a great product is that it is a lot harder than you expect for people to actually find out about and try your product. This means any avenue that increases the potential reach and distribution of your product is bringing money in your pocket. Not only does Facebook have several million active and engaged users, the Facebook platform also provides several mechanisms that encourage the viral spread of applications which developers consistently rave about. No other social networking site’s widget platform even comes close. +1 to Facebook.

    3. Access to User Data: Social networking sites are all about connecting people to the people they care about and their interests. This means that applications built on these platforms should be able to determine a user’s friends and interests to be able to give an optimal experience. The Facebook platform is unprecedented in the arena of widget platforms when it comes to the amount of user information it exposes to applications with methods like friends.getusers.getInfo, photos.get and even marketplace.getListings. +1 to Facebook.

    4. Ability to Build an Integrated and Immersive Experience: One place where the Facebook platform really changed the game is that widgets weren’t just relegated to a tiny box on the user’s profile like they are on other social networking sites but instead developers could build full blown applications that integrated fully into the Facebook experience. It’s a lot easier to keep users engaged and build non-intrusive advertising into your application if your entire application doesn’t have to fit in some 4” X 4” box on the user’s profile. +1 to Facebook.

    5. Applications Shielded from the “Winner’s Curse” of Web Development: The more successful your application becomes on the Web, the more money you have to spend on server related resources. Everyone knows the story of iLike scrambling to borrow money and servers because their Facebook application was more successful than they anticipated. Since a lot of widget developers are not richly funded startups or big companies, they may not be able to bear the costs of actually building a successful Web application without help from the platform vendor. A number of platform vendors provides hosting for static files and data storage APIs although none go as far as full blown application hosting...yet.  +0.5 to Facebook.

    From my perspective, if a social networking site’s widget platform doesn’t meet all criteria, then it can’t be considered a real competitor to Facebook’s platform. And as a developer if I had to choose between that platform and Facebook’s, there would be no choice to make.

    Now if you can afford multiple development efforts building widgets/applications for several disparate social networking site platforms, the list above is a good starting point for prioritizing the which social networking site’s to build widgets/applications for first.   

    Now playing: T.I. - U Don't Know Me


    Categories: Platforms

    October 17, 2007
    @ 04:00 AM

    It looks like my favorite team in Windows Live scored another hit with their recent release. You can see it for yourself by going to The product team has a huge list of the new changes in their post Live Search Maps v2 is out! Gemini Launches which list a number of significant features including

      • generic directions to a specific location (e.g. if coming from I-5 North vs. if coming from I-5 South) without a specific starting point. I believe this is formally called 1–click directions.
      • the ability to have the directions route around bad traffic
      • the ability to import GeoRSS, GPX and KML files as collections
      • increased metadata about businesses from partner companies like InfoUSA, CitySearch, and Gayot.

    Surprisingly, my favorite features of the new release don’t seem to have been mentioned. The removal of dual search boxes along with some of the other user interface niceties such as suggesting “Work” or “Home” when a user attempts to save a location are just great.

    There was also a bug in the driving directions to my house from Redmond which was also fixed in this release. I never even got around to mentioning it to the team. Now that’s what I call service.  

    Now playing: Lil Wayne - Go DJ (Remix) (feat. Slim Thug & Mannie Fresh)


    Categories: Windows Live

    Disclaimer: Although I work on the What’s New feed in Windows Live Spaces this should not be considered an announcement or precursor to an announcement of upcoming features of any Windows Live service.

    I spend a lot of my time these days thinking about digital lifestyle aggregators such as Facebook and FriendFeed. One of the things I wonder about is how to make them more relevant to users as a way to stay connected to each other without seeming confusing, overwhelming or just plain spammy.

    For instance, I look at the Facebook News Feed as the first significant implementation of this concept to hit the mainstream and I try to see what we can learn from their mistakes and where there is room for improvement. Below are two mistakes and one place I see room for improvement in the news feed as currently implemented by Facebook. The screenshot below is provided as a reference point.

    facebook news feed

    Wall Posts

    I’m now quite convinced that having wall posts show up in the news feed is a mistake. In general, Facebook already indulges in bad design by having a Wall-to-Wall posting which means that you can be viewing a friend’s wall and may only see one half of the conversation. So there is always a confusing loss of context when reading a wall on Facebook. This loss of context is exacerbated by adding wall posts to the news feed since now we not only have to deal with hearing one side of a conversation. Instead a user logs in and is confronted with a statement from the middle of a conversation, clicks through and only sees half the conversation, tries to click through to that and may not have access to both user’s walls.

    More than once I’ve logged into Facebook and been confronted with wall posts that would have been embarassing to the posters if they realized that their banter on some person’s guestbook wall was being broadcast out of context to all their co-workers, their manager and even their VP/CxO via the news feed.

    Application Installations

    Although I don’t agree with Kara Swisher for criticizing Facebook applications as being mostly trivial time wasters instead of professional tools, I do agree with her that most apps on the site aren’t of value. This means that using up my screen real estate to tell me that a buddy has installed the Pink Ribbon application or the ProductPulse application is spam almost every single time you do it. One should also consider that Facebook limits the amount of updates from your friends they show in your news feed to ensure a good mix of updates. I suspect most users would gladly trade the slots taken up by notifications of application installs for more personally relevant updates from their social network.

    The notifications about application installs showing up in the news feed is nice for developers but I question it’s value to users. Especially when you consider that Facebook applications already have ways to spread virally via application requests which has unfortunately led to Facebook Application Fatigue by their users.

    Groups and Events

    Today, a lot of groups on Facebook are there primarily as a way to declare affiliation as opposed to being an active community of users such as you’d find on sites like MSN Groups or Yahoo! Groups. I joined groups like I Dont care How Comfortable Crocs Are, You Look Like A Dumbass, I Am Fluent in Sarcasm, and If Wikipedia Says It, It Must Be True because I thought joining them would look funny on my friend’s news feeds as opposed to wanting to be part of these groups either as a lurker or as a regular discussion participant.

    One question is why Facebook uses the news feed to drive user to user traffic but not user to group traffic besides the “Dare has joined People Who Always Have To Spell Their Names For Other People” which encourages people to join the group but doesn’t encourage them to participate in the group. It may be that they don’t want users creating online groups within the site like you find in services like MSN Groups or Yahoo! Groups or it could just be that their platform can’t support that scale of activity. I wonder…

    The same questions apply for events as well. It would be cool if after I was invited to an event, I also got news feed updates via my news feed later on telling me if my friends were attending or that the event was getting lots of attendees which may influence my attendance. Again, you have to wonder why such obvious enhancements haven’t made it into their service.

    Now playing: Three 6 Mafia - Late Nite Tip


    Mary Jo Foley has a delightful post entitled Are all ‘open’ Web platforms created equal? where she wonders why there is more hype around the Facebook platform, Google’s muched hyped attempt to counter it on November 5th and other efforts that Anil Dash has accurately described as the new Blackbird as opposed to open API efforts from Microsoft. She posits two theories which are excerpted below

    Who isn’t mentioned in any of these conversations? Microsoft. Is it because Microsoft hasn’t opened up its various Windows Live APIs to other developers? Nope. Microsoft announced in late April its plans for opening up and providing licensing terms for several of its key Windows Live APIs, including Windows Live Contacts, Windows Live Spaces Photo Control and Windows Live Data Protocols.

    So why is Microsoft seemingly irrelevant to the conversation, when it comes to opening up its Web platform? There are a few different theories.

    “I think the excitement about the Facebook platform stems from the fact that it addresses the problem of building publicity and distribution for a new application. Any developer can create an application for Facebook, and the social network will help propagate that application, exposing it to new users,” said Matt Rosoff, an analyst with Directions on Microsoft.

    Microsoft, for its part, believes it is offering Web platform APIs the way that developers want, making them available under different business terms and permitting third parties to customize them inside their own sites, according to George Moore, General Manager of Windows Live. But Moore also acknowledges Microsoft has a different outlook in terms of which data it exposes via its APIs.

    “Facebook gives you access to your social-graph (social-networking) data. We don’t do that. We have a gallery that allows users to extend Live Spaces,” Moore said.

    Moore declined to comment on when or if Microsoft planned to allow developers to tap directly into user’s social-graph data like Facebook has done.

    I see GeorgeM all the time, so I doubt he’ll mind if I clarify his statement above since it gives the wrong impression of our efforts given the context in which it was placed. If we go back to the definition of a social graph it’s clear that what is important is that it is a graph of user relationships not one that is tied to a particular site or service. From that perspective the Windows Live Contacts API which provides a RESTful interface to the contents of a user’s Windows Live address book complete with the list of tags/relationship types the user has applied to these contacts (e.g. “Family”, “Friends”, “Coworkers”, etc) as well as which of these contacts are the user’s buddies in Windows Live Messenger is a social graph API. 

    On the other hand, this API does not give you access to the user’s Spaces friends list.  My assumption is that Mary Jo’s questions were specific to social networking sites which is why George gave that misleading answer. In addition, Yaron is fond of pointing out to me that the API is in alpha so there is still a lot that can change from now until we stamp it as v1. Until then, I’ll also decline to comment on any future plans.

    As for the claim made by Matt Rosoff, I tend to agree with his assertion that the viral propagation of applications via the Facebook’s social graph is attractive to developers. However this attractiveness comes with the price of both the users and developers being locked in Facebook’s trunk.

    I personally believe that the Web is the platform and this philosophy shines through in the API efforts at Microsoft. It may be that this is not as attractive to developers today as it should be but eventually the Web will win. Everyone who has fought the Web has lost. Facebook will not be an exception.

    Now playing: Tony Yayo - I Know You Dont Love Me (feat. G-Unit)


    Categories: Platforms | Windows Live

    Erick Schonfeld from TechCrunch writes in his post Windows Live SkyDrive Doubles Storage to 1GB, Still Can’t Keep Up With Gmail that

    Microsoft doubled the online storage consumers can get for free in Windows Live SkyDrive. It’s hard to get excited about that when Gmail is already giving me 2.9 GB of storage, with more on the way—4GB by the end of the month, and 6GB by early January, according to one estimate.

    You’d think that someone who works as a pseudo-journalist on a popular technology website would be able to tell the difference between an email service and a file storage service. You’d think he’d want to compare apples to apples and compare GMail’s 2.9 GB of Storage with Windows Live Hotmail’s 5 Gigabytes of Storage or compare the capabuilities of Microsoft’s SkyDrive with Google’s GDrive. 

    Except Google hasn’t figured out how to ship GDrive for over 5 years so it would be an apples to vaporware comparison. Smile

    Much love to my SkyDrive peeps on their new release. The champagne and ice cream yesterday was much appreciated. You can learn more about their release from the post Updates to Windows Live SkyDrive! on their team blog.  

    Now playing: Foo Fighters - My Hero


    Via Robert Scoble, I stumbled on Kara Swisher’s post entitled The Children’s Hour: Facebook Apps Are for Toddlers (There, We Said It) which has the following gem which is excerpted below

    But, so far, as popular as those apps have become, what Zuckerberg and the widget-makers have wrought is mostly silly, useless and time-wasting and the kazillion users of these widgets are pretty much just acting like little children.

    I never thought I would call the often frivolous AOL back in the day–very simply, a Neanderthal version of Facebook–a mature offering in comparison.

    While I will admit when I am not chewing nails that a lot of these apps are somewhat fun, I can’t help but ask myself that lyric from the old Peggy Lee classic: “Is that all there is?”

    This is like criticizing the Incredible Hulk for being green. You sound more ridiculous than what you are criticizing. The fact is Facebook is a site that was designed as an online version of the places where college students kill time in between classes. It’s a virtual place for “chilling in front of the library” or “maxing in front of the frat house”, so what do you expect from applications targetted at its users?

    Next thing you know, Kara Swisher will be pointing out that most of the videos on YouTube aren’t worthy of being nominated for any academy awards in the short film category and that most of the widgets on MySpace are time wasting frivolities gadgets instead of professional tools. Wink .  

    If you want a sophisticated widget platform that is intended for knowledge workers and business professionals, I’ve heard SalesForce AppExchange is exactly what you are looking for. Enjoy.

    Now playing: Fabolous - Baby Don't Go (feat. Jermaine Dupri & T-Pain)


    Categories: Social Software

    Two things I worked on over the past year or so shipped today. The first is Windows Live Events. You can learn more about in the post entitled Introducing Windows Live Events and new Windows Live Spaces updates by Chris Keating and Jay Fluegel which reads

    Easily create a great-looking website for your next event
    To offer you more ways to connect and share memories with the people you care about most, the team that brought you Spaces would love to hear your feedback on Windows Live Events, our new, free social event planning service.  With Events you can easily:

    • Plan that next baby shower, birthday party, or family reunion
    • Create a great-looking event invitation and website using one of over 100 fantastic templates
    • Invite anyone with an e-mail address and track who’s coming
    • Make your event unique with familiar customization features - choosing a friendly web address (like, using custom colors, fonts, and background images, or adding modules and Windows Live web gadgets
    • Let guests and organizers share photos and stories before and after the event

    Click on Events from the new navigation and then click Create event to get started!

    My contribution to this release was working on modifying aspects of our contacts and storage platform to understand the concept of groups of users that can be treated as a single entity [especially with regards to joint ownership of objects, sharing and access control lists] instead of being centered on a single user. I expect that Windows Live Events will be just the first of many ways in which this capability will manifest itself across Windows Live.

    Unfortunately, I didn't work on the final stage of getting the platform ready for the product to ship. Instead I went on to work on my next feature that shipped today while Ali took over working on the platform support for Windows Live Events including cleaning up my design hacks doing a better job of future proofing the design than I did. Mad props to Bob Bauman, Mike Newman, Jason Antonelli, John Kountz, Lalit Jina, Neel Malik, Mike Torres and everyone else who worked on this release across Windows Live. You guys rock.

    The second thing I was a part of building which shipped today is the updated “What’s New” page in Windows Live Spaces which is also described in detail in the aforementioned post by Chris Keating and Jay Fluegel . Before you say anything, Yes, its re-design has been influenced by the Facebook News Feed feature. Below is a screen shot of the old design of the page from the previous release

    and now contrast that with the new version of the page

    I'm pretty jazzed about getting to work on this feature since it is something I've wanted do for a quite some time. A few years ago, I remember talking to Maya about building a “friends page” similar to the Live Journal friends page in MSN Spaces and at the time the response was that I was requesting that we merge the functionality of an RSS reader with a blogging/social networking site which was at cross purposes. In hindsight, I realize that although the idea was a good one, the implementation I was suggesting was kind of hokey. Then Facebook shipped the News Feed and it all clicked.

    I worked with a lot of great folks on this feature. Paul Ming, Deepa Chandramouli, Rob Dolin, Vanesa Polo Dominguez, Jack Liu, Eric Melillo and a bunch of others who I may have failed to mention but still deserve lots of praise. This feature was the most fun I've had working in Windows Live. Not only did I get a deeper appreciation of designing for scalability but I also got to see what it is like to be responsible for components on the live site. All I need now is a pager and I'm good to go. :)

    I'd be remiss in my duties if I didn't point out that in the second screen shot, the first item on the What's New page is less than 5 minutes old. If you use other systems that have similar features, you may have noticed a much longer delay than a few minutes from posting to showing up in your news feed. As the saying goes, a lot of effort went into making this look effortless.

    I also noticed some initial feedback on this feature in the blog post by Jamie Thomson entitled new spaces home page where he writes

    There's a lot of potential for this activity list given that it could capture any activity people commit using their Live ID. Every live property has the potential for being able to post activity on here so one day we may see notifications of:

    • change of messenger status
    • posting of photos on Live Space
    • addition of gadgets to Live Space
    • items placed for sale on Expo
    • questions asked or answered on QnA
    • collection shared from Live Maps
    • video posted on MSN video
    • changes to XBox gamer card
    • changes to Zune Social (after it launches)
    • items posted to the Live Gallery
    • an event being planned
    • purchased a song from Zune marketplace
    • posts in MSN groups (soon to be Live Groups)
    • posts to online forums (
    • downloads of public files from Skydrive

    Its all pretty good but let's be honest, this is basically a clone of of what Facebook already have. Given Facebook's popularity though Microsoft didn't really have a choice but to copy them. If Microsoft really want to differentiate themselves in this arena then one option would be to provide avenues for interacting with other online services such as Flickr, Twitter, Jaiku, Pownce,  etc... This list could then become an aggregator for all online activity and that's a pretty compelling scenario. One really quick win in this area would be to capture any blog entry that is posted from Live Writer, regardless of whether it is posted to Live Spaces or not.

    Posting of photos already shows up on the "what's new" page. Downloads of files will likely never show up for privacy reasons, I'm sure you can guess why it may not be wise to broadcast what files you were downloading from shared folders to all your IM buddies and the people friends list if you think about it a little. As for the rest of the request, thanks for the feedback. We'll keep it in mind for future releases. Wink

    PS: If you work for a Microsoft property that would like to show up on the "what's new" page, host it or just wanna plain chat about the feature then give me a shout if interested in the platform or holler at Rob if it's about the user experience.


    Categories: Windows Live

    I recently got an email from a developer about my post Thoughts on Amazon's Internal Storage System (Dynamo) which claimed that I seem to be romanticizing distributed systems that favor availability over consistency. He pointed out that although this sounds nice on paper, it places a significant burden on application developers. He is 100% right. This has been my experience in Windows Live and I’ve heard enough second hand to assume it is the experience at Amazon as well when it comes to Dynamo.

    I thought an example of how this trade-off affects developers would be a useful excercise and may be of interest to my readers. The following example is hypothetical and should not be construed as describing the internal architectures of any production systems I am aware of.

    Scenario: Torsten Rendelmann, a Facebook user in Germany, accepts a friend request from Dare Obasanjo who is a Facebook user in the United States.

    The Distributed System: To improve the response times for users in Europe, imagine Facebook has a data center in London while American users a serviced from a Data Center in Palo Alto. To achieve this, the user database is broken up in a process commonly described as shardingThe question of if and how data is replicated across both data centers isn’t relevant to this example.

    The application developer who owns the confirm_friend_request() method, will ideally want to write code that took the following form 

    public void confirm_friend_request(user1, user2){

      update_friend_list(user1, user2, status.confirmed); //palo alto
      update_friend_list(user2, user1, status.confirmed); //london

    Yay, distributed transactions. You have to love a feature that every vendor advises you not to use if you care about performance. So obviously this doesn’t work for a large scale distributed system where performance and availabilty are important.  

    Things get particularly ugly when you realize that either data center or the specific server a user’s data is stored on could be unreachable for a variety of reasons (e.g. DoS attack, high seasonal load, drunk sys admin tripped over a power cord, hard drive failure due to cheap components, etc).

    There are a number of options one can consider when availability and high performance are considered to be more important than data consistency in the above example. Below are three potential implementations of the code above each with it’s own set of trade offs.

    OPTION I: Application developer performs manual rollback on error

    public void confirm_friend_request_A(user1, user2){

       update_friend_list(user1, user2, status.confirmed); //palo alto 
     }catch(exception e){ 

      update_friend_list(user2, user1, status.confirmed); //london
     }catch(exception e) {
      revert_friend_list(user1, user2);


    The problem here is that we don’t handle the case where revert_friend_list() fails. This means that Dare (user1) may end up having Torsten (user2) on his friend list but Torsten won’t have Dare on his friend list. The database has lied.

    OPTION II: Failed events are placed in a message queue to be retried until they succeed.   

    public void confirm_friend_request_B(user1, user2){

       update_friend_list(user1, user2, status.confirmed); //palo alto 
     }catch(exception e){ 
      add_to_retry_queue(operation.updatefriendlist, user1, user2, current_time()); 

      update_friend_list(user2, user1, status.confirmed); //london
     }catch(exception e) {
      add_to_retry_queue(operation.updatefriendlist, user2, user1, current_time());  


    Depending on how long the error exists and how long it takes an item to sit in the message queue, there will be times when the Dare (user1) may end up having Torsten (user2) on his friend list but Torsten won’t have Dare on his friend list. The database has lied, again.

    OPTION III: System always accepts updates but application developers may have to resolve data conflicts later. (The Dynamo approach)

    /* update_friend_list always succeeds but may enqueue an item in message queue to try again later in the event of failure. This failure is not propagated to callers of the method.  */

    public void confirm_friend_request_C(user1, user2){
       update_friend_list(user1, user2, status.confirmed); // palo alto
       update_friend_list(user2, user1, status.confirmed); //london 


    /* get_friends() method has to reconcile results returned by get_friends() because there may be data inconsistency due to a conflict because a change that was applied from the message queue is contradictory to a subsequent change by the user.  In this case, status is a bitflag where all conflicts are merged and it is up to app developer to figure out what to do. */ 

      public list get_friends(user1){ 
          list actual_friends = new list();
          list friends = get_friends();  

          foreach (friend in friends){     

            if(friend.status == friendstatus.confirmed){ //no conflict

            }else if((friend.status &= friendstatus.confirmed) 
                       and !(friend.status &= friendstatus.deleted)){

              // assume friend is confirmed as long as it wasn’t also deleted
              friend.status = friendstatus.confirmed;              
              update_friends_list(user1, friend, status.confirmed);

            }else{ //assume deleted if there is a conflict with a delete
              update_friends_list( user1, friend, status.deleted)


       return actual_friends;

    These are just a few of the many approaches that can be implemented in such a distributed system to get around the performance and availability implications of using distributed transactions. The main problem with them is that in every single case, the application developer has an extra burden placed on his shoulders due to inherent fragility of distributed systems. For a lot of developers, the shock of this realization is akin to the shock of programming in C++ after using C# or Ruby for a couple of years. Manual memory management? Actually having to perform bounds checking arrays? Being unable to use decades old technology like database transactions?

    The challenge in building a distributed storage system like BigTable or Dynamo is in balancing the need for high availability and performance while not building a system that encourages all sorts of insidious bugs to exist in the system by design.  Some might argue that Dynamo goes to far in the burden that it places on developers while there are others that would argue that it doesn’t go far enough.

    In what camp do you fall?

    Now playing: R. Kelly - Rock Star (feat. Ludacris & Kid Rock)


    Categories: Platforms | Programming | Web Development

    Yaron Goland has an entertaining post entitled Interoperability Wars - Episode 6 - Part 1 - Revenge of Babble about some of the philosophical discussions we’ve been having at work about the Atom Publishing Protocol (RFC 5023). The entire post is hilarious if you are an XML protocol geek and I recommend reading it. The following excerpt is a good starting point for another discussion about APP’s suitability as a general purpose editing protocol for the Web. Yaron writes

    Emperor Babble: Excellent, Weasdel's death will serve us well in lulling the forces of interoperability into thinking they are making progress. Welcome Restafarian, it is time you understood your true place in my plans.

    Luke Restafarian: Do what you want to me emperor, but the noble cause of interoperability will prevail!

    The Emperor turns to the center of the chamber where a form, almost blinding in its clarity, takes shape:

    GET /someuser/profile HTTP/1.1
    content-type: application/xml

    <profile xmlns="">







    Darth Sudsy is momentarily taken aback from the appearance of the pure interoperable data while Luke Restafarian seems strengthened by it. The Emperor turns back to Luke and then speaks.

    Emperor Babble: I see it strengthens you Restafarian, no matter. But look again at your blessed interoperability.

    When the Emperor turns back to the form the form begins to morph, growing darker and more sinister:

    GET /someuser/profileFeed HTTP/1.1
    content-type: application/ATOM+xml

    <feed xmlns="">
    <category scheme="" term="Profile"/>
    <content type="Application/XML" xmlns:e="">

    <content type="Application/XML" xmlns:e="">


    <content type="Application/XML" xmlns:e="">



    Luke, having recognized the syntax, clearly expects to get another wave of strength but instead he feels sickly. The emperor, looking slightly stronger, turns from the form to look at Luke.

    Luke Restafarian: What have you done? That structure is clearly taken from Princess Ape-Pea's system, she is a true follower of interoperability, I should be getting stronger but somehow it's making me ill.

    Emperor Babble: You begin to understand young Restafarian. Used properly Princess Ape-Pea's system does indeed honor all the qualities of rich interoperability. But look more carefully at this particular example of her system. Is it not beautiful? Its needless complexity, its useless elements, its bloated form, they herald true incompatibility. No developer will be able to manually deal with such an incomprehensible monstrosity. We have taken your pure interoperable data and hidden it in a mud of useless scaffolding. Princess Ape-Pea and your other minions will accept this poison as adhering to your precious principles of interoperability but in point of fact by turning Princess Ape-Pea's system into a generic structured data manipulation protocol we have forced the data to contort into unnatural forms that are so hard to read, so difficult to understand that no programmer will ever deal with it directly. We will see no repeat of those damned Hit-Tip Knights building their own stacks in a matter of hours in order to enable basic interoperability. In this new world even the most trivial data visualizations and manipulations become a nightmare. Instead of simple transversals of hierarchical structures programmers will be forced into a morass of categories and artificial entry containers. Behold!

    Yaron’s point is that since Atom is primarily designed for representing streams of microcontent, the only way to represent other types of data in Atom is to tunnel them as custom XML formats or proprietary extensions to Atom. At this point you’ve added an unnecessary layer of complexity.

    The same thing applies to the actual Atom Publishing Protocol. The current design requires clients to use optimistic concurrency to handle conflicts on updates which seems like unnecessary complexity to push to clients as opposed to a “last update wins” scheme. Unfortunately, APP’s interaction model doesn’t support granular updates which means such a scheme isn’t supported by the current design. A number of APP experts have realized this deficiency as you can see from James Snell of IBM’s post entitled Beyond APP - Partial updates and John Panzer of Google’s post entitled RESTful partial updates: PATCH+Ranges.

    A potential counter argument that can be presented when pointing these deficiencies of the Atom Publishing Protocol when used outside it’s core scenarios of microcontent publishing is that Google exposes all their services using GData which is effectively APP. This is true, but there are a couple of points to consider

    1. Google had to embrace and extend the Atom format with several proprietary extensions to represent data that was not simply microcontent.
    2. APP experts do not recommend embracing and extending the Atom format the way Google has done since it obviously fragments interoperability. See Joe Gregorio’s post entitled In which we narrowly save Dare from inventing his own publishing protocol and James Snell’s post entitled Silly for more details. 
    3. Such practices leads to a world where we have applications that can only speak Google’s flavor of Atom. I remember when it used to be a bad thing when Microsoft did this but for some reason Google gets a pass.
    4. Google has recognized the additional complexity they’ve created and now ship GData client libraries for several popular platforms which they recommend over talking directly to the protocol. I don’t know about you but I don’t need a vendor specific client library to process RSS feeds or access WebDAV resources, so why do I need one to talk to Google’s APP implementation?

    It seems that while we weren’t looking, Google move us a step away from a world of simple, protocol-based interoperability on the Web to one based on running the right platform with the right libraries

    Usually I wouldn’t care about whatever bad decisions the folks at Google are making with their API platform. However the problem is that it sends out the wrong message to other Web companies that are building Web APIs. The message that it’s all about embracing and extending Internet standards with interoperability being based on everyone running sanctioned client libraries instead of via simple, RESTful protocols is harmful to the Internet. Unfortunately, this harkens to the bad old days of Microsoft and I’d hate for us to begin a race to the bottom in this arena.

    On the other hand, arguing about XML data formats and RESTful protocols are all variations of arguments about what color to paint the bike shed. At the end of the day, the important things are (i) building a compelling end user service and (ii) exposing compelling functionality via APIs to that service. The Facebook REST APIs are a clusterfuck of inconsistency while the Flickr APIs are impressive in how they push irrelevant details of the internals of the service into the developer API (NSIDs anyone?). However both of these APIs are massively popular.

    From that perspective, what Google has done with GData is smart in that by standardizing on it even though it isn’t the right tool for the job, they’ve skipped the sorts of ridiculous what-color-to-paint-the-bike-shed discussions that prompted Yaron to write his blog post in the first place. Wink

    With that out of the way they can focus on building compelling Web services and exposing interesting functionality via their APIs. By the way, am I the last person to find out about the YouTube GData API?.

    Now playing: DJ Khaled - I'm So Hood (feat. T-Pain, Trick Daddy, Rick Ross & Plies)


    Categories: Platforms | XML Web Services

    Jenna has uploaded pictures from our wedding weekend in Las Vegas and our honeymoon in Puerto Vallarta to her Windows Live space. Below are a couple of entry points into the photo stream.

    Signing the marriage licence

    The blushing bride

    On the way to the after party

    One of the many amazing sunsets we saw in Mexico

    We got to release baby turtles into the ocean

    Nothing beats a pool with a beach view

    These are the pictures that we took ourselves. The pics from the professionals capturing the wedding day and reception will show up in a couple of weeks. 

    Now playing: Jagged Edge - Let's Get Married (remix) (feat. Run DMC)


    Categories: Personal

    Werner Vogels, CTO of Amazon, has a blog post entitled Amazon's Dynamo which contains the HTML version of an upcoming paper entitled Dynamo: Amazon’s Highly Available Key-value Store which describes a highly available, distributed storage system used internally at Amazon. 

    The paper is an interesting read and a welcome addition to the body of knowledge about building megascale distributed storage systems. I particularly like that it isn’t simply another GFS or BigTable, but is unique in it’s own right. Hopefully, this will convince folks that just because Google were first to publish papers about their internal infrastructure doesn’t mean that what they’ve done is the bible of building megascale distributed systems. Anyway, on to some of the juicy bits

    Traditionally production systems store their state in relational databases. For many of the more common usage patterns of state persistence, however, a relational database is a solution that is far from ideal. Most of these services only store and retrieve data by primary key and do not require the complex querying and management functionality offered by an RDBMS. This excess functionality requires expensive hardware and highly skilled personnel for its operation, making it a very inefficient solution. In addition, the available replication technologies are limited and typically choose consistency over availability. Although many advances have been made in the recent years, it is still not easy to scale-out databases or use smart partitioning schemes for load balancing.

    Although I work for a company that sells a relational database product, I think it is still fair to say that there is a certain level of scale where practically every feature traditionally associated with an RDBMS works against you.

    Luckily, there are only a handful of companies and Web services in the world that need to operate at that scale.

    2.1 System Assumptions and Requirements

    The storage system for this class of services has the following requirements:

    Query Model: simple read and write operations to a data item that is uniquely identified by a key. State is stored as binary objects (i.e., blobs) identified by unique keys. No operations span multiple data items and there is no need for relational schema. This requirement is based on the observation that a significant portion of Amazon’s services can work with this simple query model and do not need any relational schema. Dynamo targets applications that need to store objects that are relatively small (usually less than 1 MB).

    ACID Properties: ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties that guarantee that database transactions are processed reliably. In the context of databases, a single logical operation on the data is called a transaction. Experience at Amazon has shown that data stores that provide ACID guarantees tend to have poor availability. This has been widely acknowledged by both the industry and academia [5]. Dynamo targets applications that operate with weaker consistency (the “C” in ACID) if this results in high availability. Dynamo does not provide any isolation guarantees and permits only single key updates.

    Other Assumptions: Dynamo is used only by Amazon’s internal services. Its operation environment is assumed to be non-hostile and there are no security related requirements such as authentication and authorization. Moreover, since each service uses its distinct instance of Dynamo, its initial design targets a scale of up to hundreds of storage hosts. We will discuss the scalability limitations of Dynamo and possible scalability related extensions in later sections.

    Lots of worthy items to note here. The first is that you can get a lot of traction out of a simple data structure such as a hash table. Specifically, as noted by Sam Ruby in his post Key + Data, accessing data by key instead of using complex queries is becoming a common pattern in large scale distributed storage systems. Sam actually missed pointing out that Google’s Bigtable is another example of this trend given that data items within it are accessed using the tuple {row key, column key, timestamp} instead of being queried using data manipulation language.

    Another interesting thing, from my perspective, is that they’ve gotten around hitting scaling limits at running it on hundreds of storage hosts by having different teams at Amazon run their own instances of Dynamo. Then again, there are 200 clusters of GFS running at Google, so this is probably common sense as well.

    4.4 Data Versioning

    Dynamo provides eventual consistency, which allows for updates to be propagated to all replicas asynchronously. A put() call may return to its caller before the update has been applied at all the replicas, which can result in scenarios where a subsequent get() operation may return an object that does not have the latest updates.. If there are no failures then there is a bound on the update propagation times. However, under certain failure scenarios (e.g., server outages or network partitions), updates may not arrive at all replicas for an extended period of time.

    There is a category of applications in Amazon’s platform that can tolerate such inconsistencies and can be constructed to operate under these conditions. For example, the shopping cart application requires that an “Add to Cart” operation can never be forgotten or rejected. If the most recent state of the cart is unavailable, and a user makes changes to an older version of the cart, that change is still meaningful and should be preserved. But at the same time it shouldn’t supersede the currently unavailable state of the cart, which itself may contain changes that should be preserved. Note that both “add to cart” and “delete item from cart” operations are translated into put requests to Dynamo. When a customer wants to add an item to (or remove from) a shopping cart and the latest version is not available, the item is added to (or removed from) the older version and the divergent versions are reconciled later.

    In order to provide this kind of guarantee, Dynamo treats the result of each modification as a new and immutable version of the data. It allows for multiple versions of an object to be present in the system at the same time. Most of the time, new versions subsume the previous version(s), and the system itself can determine the authoritative version (syntactic reconciliation). However, version branching may happen, in the presence of failures combined with concurrent updates, resulting in conflicting versions of an object. In these cases, the system cannot reconcile the multiple versions of the same object and the client must perform the reconciliation in order to collapse multiple branches of data evolution back into one (semantic reconciliation). A typical example of a collapse operation is “merging” different versions of a customer’s shopping cart. Using this reconciliation mechanism, an “add to cart” operation is never lost. However, deleted items can resurface.

    Fascinating. I can just imagine how scary this most sound to RDBMS heads to think that instead of the database enforcing the rules of consistency, it just keeps multiple versions of the “row” around and then asks the client to figure out which is which if there were multiple updates that couldn’t be reconciled.

    The folks at Amazon have taken acknowledgement of the CAP Conjecture to its logical extreme. Consistency, Availability, and Partition-tolerance. Pick two.

    There’s lots of other interesting stuff in the paper but I’ll save some for you to read and end my excerpts here. This will make great bedtime reading this weekend.

    Now playing: Geto Boys - My Mind's Playin Tricks On Me



    According to blog posts like A Flood of Mashups Coming? OAuth 1.0 Released and John Musser’s OAuth Spec 1.0 = More Personal Mashups? , it looks like the OAuth specification has reached it’s final draft.

    This is good news because the need for a standardized mechanism for users to give applications permission to access their data or act on their behalf has been obvious for a while. The most obvious manifestation of this are all the applications that ask for your username and password so they can retrieve your contact list from your email service provider.

    So what exactly is wrong with applications like the one’s shown below?



    The problem with these applications [which OAuth solves] is that when I give them my username and password, I’m not only giving them access to my address book but also access to

    because all of those services use the same credentials. Sounds scary when put in those terms doesn’t it?

    OAuth allows a service provider (like Google or Yahoo!) to expose an interface that allows their users to give applications permission to access their data while not exposing their login credentials to these applications. As I’ve mentioned in the past, this standardizes the kind of user-centric API model that is utilized by Web services such as the Windows Live Contacts API, Google AuthSub and the Flickr API to authenticate and authorize applications to access a user’s data.  

    The usage flow end users can expect from OAuth enabled applications is as follows.

    1. The application or Web site informs the user that it is about to direct the user to the service provider’s Web site to grant it permission.

    2. The user is then directed to the service providers Web site with a special URL that contains information about the requesting application. The user is prompted to login to the service provider’s Website to verify their identity. 


    3. The user grants the application permission.

    4. The application gets access to the user’s data and the user never had to hand over their username and password to some random application which they might not trust.

    I’ve read the final draft of the OAuth 1.0 spec and it seems to have done away with some of the worrisome complexity I’d seen in earlier draft (i.e. single use and multi-use tokens). Great work by all those involved.

    I never had time to participate in this effort but it looks like I wouldn’t have had anything to add. I can’t wait to see this begin to get deployed across the Web.

    Now playing: Black Eyed Peas - Where is the Love (feat. Justin Timberlake)


    Scott Guthrie has a blog post entitled Releasing the Source Code for the .NET Framework Libraries where he writes

    One of the things my team has been working to enable has been the ability for .NET developers to download and browse the source code of the .NET Framework libraries, and to easily enable debugging support in them.

    Today I'm excited to announce that we'll be providing this with the .NET 3.5 and VS 2008 release later this year.

    We'll begin by offering the source code (with source file comments included) for the .NET Base Class Libraries (System, System.IO, System.Collections, System.Configuration, System.Threading, System.Net, System.Security, System.Runtime, System.Text, etc), ASP.NET (System.Web), Windows Forms (System.Windows.Forms), ADO.NET (System.Data), XML (System.Xml), and WPF (System.Windows).  We'll then be adding more libraries in the months ahead (including WCF, Workflow, and LINQ).  The source code will be released under the Microsoft Reference License (MS-RL).

    This is one of those announcements I find hard to get excited about. Any developer who’s been frustrated by the weird behavior of a .NET Framework class and has wanted to look at it’s code, should already know about Lutz Roeder’s Reflector which is well known in the .NET devoper community. So I’m not sure who this anouncement is actually intended to benefit.

    On the other hand, I’m sure Java developers are having a chuckle at our expense that it took this long for Microsoft to allow developers to see the source code for ArrayList.Count so we can determine if it is lazily or eagerly evaluated.

    Oh well, better late than never.

    PS: The ability to debug into .NET Framework classes will be nice. I’ve wanted this more than once while working on RSS Bandit and will definitely take advantage of it if I ever get around to installing VS 2008.

    Now playing: TLC - Somethin Wicked This Way Comes (feat. Andre 3000)


    Categories: Programming

    This is another post I was planning to write a few weeks ago which got interrupted by my wedding and honeymoon.

    A few weeks ago, Joel Spolsky wrote a post entitled Strategy Letter VI which I initially dismissed as the ravings of a desktop developer who is trying to create an analogy when one doesn’t exist. The Web isn’t the desktop, or didn’t he read There is no Web Operating System (or WebOS)? By the second time I read it, I realized that if you ignore some of the desktop-centric thinking in Joel’s article, then not only is Joel’s article quite insightful but some of what he wrote is already coming to pass.

    The relevant excerpt from Joel’s article is

    Somebody is going to write a compelling SDK that you can use to make powerful Ajax applications with common user interface elements that work together. And whichever SDK wins the most developer mindshare will have the same kind of competitive stronghold as Microsoft had with their Windows API.

    If you’re a web app developer, and you don’t want to support the SDK everybody else is supporting, you’ll increasingly find that people won’t use your web app, because it doesn’t, you know, cut and paste and support address book synchronization and whatever weird new interop features we’ll want in 2010.

    Imagine, for example, that you’re Google with GMail, and you’re feeling rather smug. But then somebody you’ve never heard of, some bratty Y Combinator startup, maybe, is gaining ridiculous traction selling NewSDK,

    And while you’re not paying attention, everybody starts writing NewSDK apps, and they’re really good, and suddenly businesses ONLY want NewSDK apps, and all those old-school Plain Ajax apps look pathetic and won’t cut and paste and mash and sync and play drums nicely with one another. And Gmail becomes a legacy. The WordPerfect of Email. And you’ll tell your children how excited you were to get 2GB to store email, and they’ll laugh at you. Their nail polish has more than 2GB.

    Crazy story? Substitute “Google Gmail” with “Lotus 1-2-3”. The NewSDK will be the second coming of Microsoft Windows; this is exactly how Lotus lost control of the spreadsheet market. And it’s going to happen again on the web because all the same dynamics and forces are in place. The only thing we don’t know yet are the particulars, but it’ll happen

    A lot of stuff Joel asserts seems pretty clueless on the face of it. Doesn’t he realize that there are umpteen billion AJAX toolkits (e.g. Dojo, Google Web Toolkit, Yahoo! User Interface Library,, etc)  and rich internet application platforms (e.g. Flash, Silverlight, XUL, etc)? Doesn’t he realize that there isn’t a snowball’s chance in hell of the entire Web conforming to standard user interface guidelines let alone everyone agreeing on using the same programming language and SDK to build Web apps?

    But wait…

    What happens if you re-read the above excerpt and substitute NewSDK with Facebook platform?

    I didn’t classify Facebook as a Social Operating System for no reason. GMail and other email services have become less interesting to me because I primarily communicate with friends and family on the Web via Facebook and it’s various platform applications. I’ve stopped playing casual games at Yahoo! Games and now use Scrabulous and Texas Hold ‘Em when I want to idle some time away on the weekend. All of these applications are part of a consistent user interface, are all accessible from my sidebar and each of them has access to my data within Facebook including my social graph. Kinda like how Windows or Mac OS X desktop applications on my machine have a consistent user interface, are all accessible from my applications menu and can all access the data on my hard drive.


    I suspect that Joel is right about NewSDK, he’s just wrong about which form it will take. “Social operating system” does have a nice ring to it, doesn’t it?

    Now playing: Kanye West - Two Words (feat. Mos Def, Freeway & The Harlem Boys Choir)


    Categories: Platforms | Web Development

    October 4, 2007
    @ 04:00 AM

    Mini-Microsoft has a blog post up to let us know that his Facebook account was cancelled. In the comments he clarifies he wasn’t specifically targetted and this is just part of the Facebook terms of service. He writes

    For those who probably will never see this Facebook help-topic, this is what I've been directed to:

    The only relevant text that I can find:

    "Facebook does not allow users to register with fake names, to impersonate any person or entity, or to falsely state or otherwise misrepresent themselves or their affiliations."

    I imagine they only do something when someone complains vs. being constantly policing things. And someone out there (scanning the crowd of exceptionally good looking people who visit here) must have taken it upon themselves to complain.

    I didn’t realize that if I don’t provide 100% accurate data about myself (thus making identity theft easier) I could get my account banned from Facebook.

    I can understand why they want to encourage people to use real names since they want to be the kind of place that have users like “Dare Obasanjo” and “Robert Scoble” not ‘carnage4life’ and ‘scobleizer’ since the former implies a more personal experience.

    However it seems dumb to be trying to replicate Friendster’s mistake by killing off every account that didn’t conform to their standards. There are ways to encourage such behavior without being jerks as they’ve clearly been in this case.

    Now playing: Dem Franchize Boyz - Oh I Think They Like Me (remix) (feat. Jermaine Dupri, Da Brat & Lil Bow Wow)


    Yesterday morning, I tossed out a hastily written post entitled It Must Be a Fun Time to Work on Microsoft Office which seems to have been misread by some folks based on some of the comments I’ve seen on my blog and in other places. So further exposition of some of the points in that post seems necessary.

    First of all, there’s the question of who I was calling stupid when talking about the following announcements

    • Google announcing the launch of Presently, their Web-based Powerpoint clone. Interestingly enough, one would have expected presentation software to be the most obvious application to move to the Web first instead of the last.
    • Yahoo! announcing the purchase of Zimbra, a developer of a Web-based office productivity and collaboration suite.
    • Microsoft announcing the it would integrate Web-based storage and collaboration into it’s desktop office productivity suite.
    • IBM announcing that it would ship it’s own branded version of an Open Source clone of Microsoft’s desktop productivity suite.

    Given that three of these announcements are about embracing the Web and the last one is about building disconnected desktop software, I assumed it was obvious who was jumping on a dying paradigm while the rest of the industry has already moved towards the next generation. To put this another way, James Robertson’s readers were right that I was talking about IBM.

    There is something I did want to call out about James Robertson’s post. He wrote

    People have moved on to the 80% solution that is the web UI, because the other advantages outweigh that loss of "richness".

    I don’t believe that statement when it comes to office productivity software. I believe that the advantages of leveraging the Web are clear. From my perspective

    1. universal access to my data from any device or platform 
    2. enabling collaboration with “zero install” requirements on collaborators

    are clear advantages that Web-based office productivity software has over disconnected desktop software.

    It should be noted that neither of these advantages requires that the user interface is Web-based or that it is rich (i.e. AJAX or Flash if it is Web-based). Both of these things help but they aren’t a hard requirement.

    What is important is universal access to my data via the Web. The reason I don’t have an iPhone is because I’m hooked on my Windows Mobile device because of the rich integration it has with my work email, calendar and tasks list. The applications on my phone aren’t Web-based, they are the equivalent of “desktop applications” for my phone. Secondly, I didn’t have to install them because they were already on my phone [actually I did have to install Oxios ToDo List but that’s only because the out-of-the-box task list synchronization in Windows Mobile 5 was less than optimal for my needs].

    I used to think that having a Web-based interface was also inevitable but that position softened once I realized that you’ll need offline support which means building support for local storage + synchronization into the application (e.g. Google Reader's offline mode) to truly hit the 80/20 point for most people given how popular laptops are these days. However once you’ve built that platform, the same storage and synchronization engine could be used by a desktop application as well.

    In that case, either way I get what I want. So desktop vs. Web-based UI doesn’t matter since they both have to stretch themselves to meet my needs. But it is probably a shorter jump to Web-enable the desktop applications than it is to offline-enable the Web applications.  

    Now playing: Playa Fly - Feel Me


    This is one of those posts I started before I went on my honeymoon and never got around to finishing. There are lots of interesting things happening in the world of office productivity software these days. Here are four announcements from the past three weeks that show just how things are heating up in this space, especially if you agree with Steve Gillmor that Office is Dead *(see footnote).

    From the article Google Expands Online Software Suite 

    MOUNTAIN VIEW, Calif. (AP) — Google Inc. has expanded its online suite of office software to include a business presentation tool similar to Microsoft Corp.'s popular PowerPoint, adding the latest twist in a high-stakes rivalry.

    Google's software suite already included word processing, spreadsheet and calendar management programs. Microsoft has been reaping huge profits from similar applications for years.

    Unlike Google's applications, Microsoft's programs are usually installed directly on the hard drives of computers.

    From the article I.B.M. to Offer Office Software Free in Challenge to Microsoft’s Line

    I.B.M. plans to mount its most ambitious challenge in years to Microsoft’s dominance of personal computer software, by offering free programs for word processing, spreadsheets and presentations.

    Steven A. Mills, senior vice president of I.B.M.’s software group, said the programs promote an open-source document format.

    The company is announcing the desktop software, called I.B.M. Lotus Symphony, at an event today in New York. The programs will be available as free downloads from the I.B.M. Web site.

    From the blog post Yahoo scoops up Zimbra for $350 million

    Yahoo has been on an acquisition binge late, but mostly to expand its advertising business. Now Yahoo is buying its way deeper into the applications business with the acquisition of Zimbra for a reported $350 million, mostly in cash. Zimbra developed a leading edge, Web 2.0 open source messaging and collaboration software suite, with email, calendar, document processing and a spreadsheet.

    and finally, from the press release Microsoft Charts Its Software Services Strategy and Road Map for Businesses

     Today Microsoft also unveiled the following:

    • Microsoft® Office Live Workspace, a new Web-based feature of Microsoft Office that lets people access their documents online and share their work with others

    Office Live Workspace: New Web Functionality for Microsoft Office

    Office Live Workspace is among the first entries in the new wave of online services. Available at no charge, Office Live Workspace lets people do the following:

    • Access documents anywhere. Users can organize documents and projects for work, school and home online, and work on them from almost any computer even one not connected to the company or school network. They can save more than 1,000 Microsoft Office documents to one place online and access them via the Web.
    • Share with others. Users can work collaboratively on a project with others in a password-protected, invitation-only online workspace, helping to eliminate version-control challenges when e-mailing drafts to multiple people. Collaborators who don’t have a desktop version of Microsoft Office software can still view and comment on the document in a browser.

    As you can see one of these four announcements is not like the others. Since it isn’t fair to pick on the stupid, I’ll let you figure out which company is jumping on a dying paradigm while the rest of the industry has already moved towards the next generation.  The Web is no longer the future of computing, computing is now about the Web.

    * I do. Disconnected desktop software needs to go the way of the dodo.

    Now playing: Prince - Sign 'O' the Times


    October 2, 2007
    @ 03:20 PM

    Over a year ago, I commented that sometimes it feels like working at Microsoft is like working in Dinosaur Country. Every time, I hear the phrase “software as a service” or it’s cousin “software plus services” it makes me feel this way. Most of the people uttering this crap don’t realize that this makes them sound as dated as the old codgers who kept on talking about “horseless carriages” when everyone else called them automobiles or just plain cars.

    Case in point, this article from the Telegraph entitled Microsoft powers up for change which contains this humdinger of an opening paragrapgh

    Chief executive says free software, downloadable online, is on the horizon for consumers. Josephine Moulds reports

    Steve Ballmer, chief executive of Microsoft, yesterday signalled another step towards a dramatic change in the software giant's business model.

    In London on a whistle-stop tour, Ballmer was discussing the delivery of software packages over the internet. "We are a software company, and yet in a sense, the very form of our core capability is changing. We need to change our capabilities so that we are not just good at writing bits that you put out on CD and deliver, but rather writing this thing that is a living, breathing, dynamic, organic thing."

    What’s next? A press release announcing that pasteurization may not be a fad? A news story conceding that heavier-than-air aircraft may just be the way to go after all? 


    Now playing: The Verve - Bitter Sweet Symphony


    Categories: Life in the B0rg Cube

    I scored an invite to FriendFeed and after trying out the service, I have to say it is both disappointing and encouraging at the same time. It is disappointing because one would expect folks like Bret Taylor and Paul Buchheit who helped launch Google Maps, Gmail and AdSense while at Google to come up with something more innovative than a knock-off of Plaxo Pulse and Google’s SocialStream which are themselves knock-offs of the Facebook News feed.

    On the other hand, this is encouraging because it is another example of how the digital lifestyle aggregator is no longer just a far out idea being tossed around on Marc Canter’s blog but instead has become a legitimate product category.  

    So what exactly is FriendFeed? The site enables users to associate themselves with the various user generated content (UGC) sites which they use regularly that publish RSS feeds or provide open APIs and then this is turned into the equivalent of a Facebook Mini Feed for the user. You can get a good idea of it by viewing my page at which aggregates the recent activities from my profiles on reddit, digg, and youtube.

    The “innovation” with FriendFeed is that instead of asking you to provide the URLs of your RSS feeds, the site figures out your RSS feed from your username on the target service. See the screenshot below for this in action

    Of course, this same “innovation” exists in Plaxo Pulse so this isn’t mindblowing. If anything, FriendFeed is currently a less feature rich version of Plaxo Pulse.

    I personally doubt that this site will catch on because it suffers from the same chicken and egg problem that face all social networking sites that depend on network effects. And if it does catch on, given that there is zero barrier to entry in the feature-set they provide, I wouldn’t be surprised to see Facebook and a host of other services roll this into their feature set. I expect that News Feed style pages will eventually show up in a majority of social sites, in much the same way that practically every website these days has a friend’s list and encourages user generated content. It’s just going to be another feature when it comes to making a website, kinda like using tabs for navigation.

    I’m sure Marc Canter finds this validation of his vision quite amusing.

    Now playing: Puddle of Mudd - Control


    October 2, 2007
    @ 04:00 AM

    I don’t really have anything to say about this that hasn’t already been said but I did find the following article in the New York Times entitled  EBay’s $4 Billion Lesson in the Value of Hype worth sharing. Juicy bits excerpted below

    As Microsoft mulls putting up to $500 million into Facebook at a $10-billion-plus valuation, it may want to consider the fate of eBay’s adventure with the Internet phone service Skype.

    When eBay bought Skype in 2005, it boasted that Skype had 52 million users and was adding 150,000 new ones a day. Even though Skype only had $60 million in revenue that year, eBay figured that with so many users it would be able to profit somehow — both by charging fees for communication services and through links to its auction and payments services. Today, eBay admitted this was a whopper of a mistake, and is taking a $1.4 billion charge to reflect the gap between what it paid for Skype and what it turns out to be worth. EBay paid $2.6 billion in cash two years ago for Skype and said it would pay up to an additional $1.5 billion based on how the company performed.


    1. Just because a company has a huge and growing audience doesn’t mean it can find a huge revenue source. Skype’s appeal is that it offers services free or very cheap. That limits its ability to raise prices. And it turns out that there are limited opportunities for advertising or add-on services.
    2. It’s almost impossible to pay for a deal through “synergies.” EBay executives talked about how Skype would be useful to connect buyers and sellers in its marketplace. This always seemed to be hooey. The eBay market is already full of chatter, mainly by e-mail, and sometimes by phone. Sure, some of that might well be handled by Internet phone, but how much and what value was created by eBay owning its own voice chat system? Not much, it turns out.

    I can't imagine any metric under which it made sense for eBay to pay over $2 billion for Skype, let alone the $4 billion which was the potential final price. This deserves to go straight to the top of the  List of the Worst Billion Dollar Internet Acquisitions of all time.

    Now playing: DJ Green Lantern - D12/50 Cent - Rap Game