In thinking about the Google OpenSocial Announcement I realized how much some of Google's recent moves remind me of Microsoft of old [for some undeclared definition of old]. We have

• Google is now a two trick pony with two self reinforcing monopolies (Search + AdSense/Adwords) which is reminiscent of the self reinforcing nature of Microsoft's Windows + Office.

• A scared overreaction to the growing hype of a startup that is eclipsing it in coolness. See OpenSocial reaction to Facebook
  

• Several high level and very visible defections to said startup from people who were highly regarded there. Seems like just yesterday we were reading similar articles and blog posts about MSFT -> Google employee defections.
  

• Said startup may be attacking Google's core monopoly in ways that were unforeseen but make sense in hindsight. See Facebook's SocialAds announcements.
  

• Creation of de jure and de facto industry standards which are pitched as being great for customers but seem suspiciously like attempts at Fire & Motion. Compare WS-* to AtomPub + GData + OAuth + OpenSocial + ???.

James Snell has a blog post entitled Batch! which talks about the Batch processing model in GData APIs. He provides a sample of a GData batch request and points out the following

If the mere sight of this doesn’t give you shivers and shakes, let me give you a few reasons why it should:

1. It’s not valid Atom. Note the first entry in the feed for instance. An Atom entry has an id, a title, some content, an author, some links, maybe some categories, etc. If the type of objects you want to represent does not also have those things, Atom is not the right format to use.
2. It only works with Atom. What about binary resources like Jpeg’s? I guess we could base64 encode the binary data and stuff that into our invalid Atom entries but doing so would suck.
3. We can’t use Etag’s and conditional requests
4. I’m sure there are more reasons but these should be enough to convince you that a better approach is needed.

In a previous post I entitled One Protocol to Rule Them All and in the Darkness Bind Them I pointed out that since the Atom Publishing Protocol is not a good fit for interacting with data types that aren’t microcontent, it would need to be embraced and extended to satisfy those needs. In addition, this leads to problems because different vendors will embrace and extend it in different ways which fragments interoperability.

An alternative approach would be for vendors to utilize protocols that are better suited for the job instead of creating incompatible versions of a standard protocol. However the response I’ve seen from various people is that it is better if we have multiple slightly incompatible implementations of a single standard than multiple completely incompatible proprietary technologies. I’ve taken to calling this “the ODF vs. OOXML lesson”. This also explains why there was so much heat in the RSS vs. Atom debates but not so much when it came to debates over Yahoo’s podcasting extensions vs. Apple’s podcasting extensions to RSS.

Let’s say we now take it as a given that there will be multiple proprietary extensions to a standard protocol and this is preferable to the alternative, what should we have as the ground rules to ensure interoperability isn’t completely thrown out the windows? A fundamental ground rule should be that vendors should actually provide standards compliant implementations of the protocol before deciding to embrace and extend it. That way clients and services that conform to the standard can interoperate with them. In this regard, GData falls down horribly as James Snell points out.

Given that Joe Gregorio now works at Google and is a co-author of RFC 5023, I assume it is just a matter of time before Google fixes this brokenness. The only question is when?

PS: Defining support for batch operations in a standard way is going to be rather difficult primarily because of how to deal with failure modes. The fact that there is always the struggle between consistency and availability in distributed systems means that some folks will want a failure in any of the batched operations to result in the equivalent of a rollback while there are others that don’t care if one or two out of a batch of fifty operations fails. Then there are some folks in the middle for whom “it depends on the context” for which kind of failure mode they want.  

Now playing: Method Man - Say (feat. Lauryn Hill)

There’s nothing like a successful company with a near monopoly to force the software industry to come up with standards. Or in this case, as in many others, force it’s competitors to band together and call what they are doing the standard because more than one vendor supports it.

From TechCrunch’s article Details Revealed: Google OpenSocial(To Launch Thursday we learn

Google wants to create an easy way for developers to create an application that works on all social networks. And if they pull it off, they’ll be in the center, controlling the network.

What They’re Launching

OpenSocial is a set of three common APIs, defined by Google with input from partners, that allow developers to access core functions and information at social networks:

• Profile Information (user data)
• Friends Information (social graph)
• Activities (things that happen, News Feed type stuff)

Hosts agree to accept the API calls and return appropriate data. Google won’t try to provide universal API coverage for special use cases, instead focusing on the most common uses. Specialized functions/data can be accessed from the hosts directly via their own APIs.

Unlike Facebook, OpenSocial does not have its own markup language (Facebook requires use of FBML for security reasons, but it also makes code unusable outside of Facebook). Instead, developers use normal javascript and html (and can embed Flash elements). The benefit of the Google approach is that developers can use much of their existing front end code and simply tailor it slightly for OpenSocial, so creating applications is even easier than on Facebook.

Similar details are available from folks like Om Malik and Marc Andreesen. 

This is a brilliant move. I’ve blogged on multiple occassions that the disparate widget platforms in social networking sites is a burden for widget developers and will lead to a “winner takes all” situation because no one wants to support umpteen different platforms. If enough momentum gains around OpenSocial, then three things will happen

• Widget developers will start to favor coding to OpenSocial because it supports multiple sites as well as targeting the Facebook platform  
• Eventually Facebook platform developers will start asking Zuckerburg and company to support OpenSocial so they only need to worry about one code base (kinda, it won’t be that easy)  
• Other companies with proprietary widget platforms or plans to create one will bow down to the tide and adopt OpenSocial

Of course, this requires a popular social networking site with a wide audience (e.g. MySpace) to adopt the platform before we see this kind of traction.

However this is the only thing Google could have done that makes any sense. Building a clone of the Facebook platform like some social networking sites planned would have been dumb because that would be the tail wagging the dog. Similarly building a competing proprietary platform would also have been dumb due to the winner takes all problem I mentioned earlier.

This is the only move that has a chance of actually giving their anti-Facebook platform a chance of being successful.

I wonder how my coworkers in Windows Live are going to take this news?

Now playing: 50 Cent - All Of Me (Feat. Mary J Blige) (Prod by Jake One)

Web companies are primarily media companies. Their job is to assemble audiences and then sell ads to their captive audiences. This is what the newspaper, television and magazine industries did before them but for some reason Web companies like Google love to claim that they aren’t media companies.  The irony is that Google stumbled on the most lucrative media property of all time (Web search results) as well as the most lucrative way of monetizing it (auction based keyword sales).

Web 1.0 was about gathering a bunch of eyeballs and then entertaining them with a bunch of content you created. Web 2.0 is all about gathering a bunch of eyeballs then having them entertain themselves with content they’ve created (also known as UGC). The most important thing hasn’t changed, it’s about gathering eyeballs.

This takes us to social networking sites like MySpace and Facebook which have taken the act of gathering eyeballs and then having your users entertain themselves with the help of random developers and elevated it to a fine art.  Things get interesting when you realize that a lot of these “random developers” have built compelling experiences and have gathered a significant audience in their own right on the backs of these social networking sites.

In a blog post entitled Scrabulous Jason Antonelli writes

I've become addicted to Scrabulous on Facebook…I'm on my 3rd game of Scrabble with my mom (yes I won two games so far) - and it usually starts with a phone call where we play for a bit and chat about various mom/son things.  My mom lives 3000+ miles away -- she's in Wilmington, NC and I'm in Seattle.  After a while when one of us needs to go do something else - we'll go ahead and play it throughout the day - maybe each making a 1-2 moves a day.

This is so much fun (though I wonder how much it would be if I didn't have a 100% win rate) and it clearly is bringing social networks into the mainstream.  The fact that this game is hosted on the Facebook platform (although it does look like it has a standalone site as well) gave two major wins to Facebook.  First I actually convinced by mom (I'm 33 so you can only guess her age!) to signup for Facebook - the first person I got to sign up to any social networking application even including the one I've worked on.  And second, both my mom and I log in to Facebook multiple times a day.

I’m similarly hooked on Scrabulous on Facebook and estimate that 75% of my page views to *.facebook.com are actually to the Scrabulous application page. I’m definitely see more ads hosted within Scrabulous than I see those hosted within Facebook*. Which leads me to wonder especially when I notice that Scrabulous now sports an ads by Google logo. I specifically wondered about the reach of the most popular widget makers and was surprised when I took a look at the Slide.com About Page which informed me that  

Slide is the largest personal media network in the world, reaching more than 134 million unique global viewers each month and 30 percent of the U.S. Internet audience. We help people express themselves and tell stories through personalized photos and videos created on Slide.com and viewed anywhere on the web or desktop.

Slide widgets — including Slideshows, Guestbooks, SkinFlix and FunPix — are popular on top social networking and blog platforms, including MySpace, Facebook, Bebo, Hi5, Friendster, Tagged, Piczo and Blogger. Slide is also the leading application developer on Facebook with more than 63 million applications installed, including SuperPoke and Top Friends, the most active application by more than four times that of any other 3rd party developer.

Whoa. I suspect that once MySpace ships their platform which relaxes the rules on advertising within widgets, companies like Slide will start to look like very interesting acquisition targets. Or maybe there’s enough money to be made generating the majority of the page views on the most popular social networking sites in the world, that there’ll be no incentive to cash out. Interesting times ahead.

* I do think I’ve clicked more on Facebook hosted ads than those hosted within Scrabulous so that may not be a useful metric.

Now playing: Fabolous - This Is My Party

Dave Winer has an interesting post where he asks Facebook app or Firefox plug-in? 

Which is a more interesting platform -- Facebook or Firefox?

This was a topic of conversation at the Web 2.0 Summit last week in SF, not on stage, but in a LobbyConversation between myself and venture capitalist Bijan Sabet.

Bijan Sabet: "I like that Firefox developers don't have to live in a world where they lie awake at night worried that the platform company is going to make life hard for them."

You can guess my answer to the question so I won’t bother answering. I will point out that I doubt that the developers of Flock would agree with the statement that Firefox doesn’t do anything to screw developers on their platform. 

I also question anyone who claims that there are more opportunities to make money off of Firefox plugins than there are selling ads in your Facebook application.

Now playing: 2Pac - Fight Music (feat. Xzibit)

Via Greg Linden I found the the summary of a recent survey 1,001 US adults conducted by Kelton Research. The summary was posted on Search Engine Land in an article entitled Report: 7 Out Of 10 Americans Experience 'Search Engine Fatigue' which states

The report discusses user frustration with clutter and the content of search results:

When asked to name their #1 complaint about the process, 25 percent cited a deluge of results, 24 percent cited a predominance of commercial (paid) listings, 18.8 percent blamed the search engine’s inability to understand their keywords (forcing them to try again), and 18.6 percent were most frustrated by disorganized/random results.

There was also a desire among many users that search engines be able to "read their minds":

Kelton asked survey respondents whether they wished that search engines like Google could, in effect, read their minds, delivering the results they were actually looking for. . . That capability is something that 78 percent of all survey-takers “wished” for, including 86.2 percent of 18-34 year-olds and 85 percent of those under 18.

That sounds like an argument for search personalization.

Search personalization is just one of the many ways to deal with end user frustrations with search results. For example, no amount of search personalization will be as effective as statistical analysis to discover and automatically fix mispellings in search queries (e.g. all the misspellings of britney spears Google has to deal with). Without spelling correction, a user will leave frustrated because they get few or no results when in truth they typed a misspelling into the search box.

The folks on the Live Search team have a first in a series of blog posts about how they tackle the problem of determining a user’s intent from their search queries entitled “Do what I mean, not what I say!” [Part 1 of 2]. It contains the following excerpt

We've been working on returning the very best search results for your intent, not just for the particular search terms that you happen to have chosen as a vehicle to transmit that intent.  There's an important difference there and it's been our focus for quite some time.

…

AutoSpell Correction

The first example of this is our new AutoSpell feature.

If we are absolutely, completely, totally, "no doubt about it" confident you misspelled one of your search terms, we automatically deliver a page that includes spell-corrected results, rather than a page of misspelled results accompanied by a "Did you mean _______?" link at the top.

For example, there's this pizza place near Microsoft called Pagliacci Pizza that is fantastic.  The problem is that I can never remember the correct spelling of the place.  My misspelled attempts are usually something along the lines of Pagliaci Pizza, Pagliaccis Pizza, or Paggliacci Pizza...

…

Stemming 

Another improvement in the "Do what I mean, not what I say" category is stemming.  Stemming means matching on the "stem" (or root) of the word rather than the exact word.

For example, users told us that the search half price book Redmond returned horrible results.  Searching for half price books Redmond produced much better results.   In our new release of Live Search, searches for half price book Redmond automatically include results with books in them as well.

Our team knew that tackling stemming would give us the improvements we needed for searches like these.  But we had to be careful, because you can't just stem all the time-you have to be smart about it.  An example of this is the word cable.  When you search for "cable," you could be looking for information on cable TV providers.  When you search for "cables", you could be looking for power, telephone, or network cables. 

These are just two techniques beyond just personalization which gives users better satisfaction with their search results and the impression that the search engine is “reading their mind”.

The main problem I have with personalization is that you need to give the search engine a private, personal information amount of information before it has tangible effects on search results. In the recent past, Marissa Mayer has pointed out that user studies have shown that location is the only significant factor which impacts perception of the relevance of search results when it comes to personalization. I suspect that true personalization will come from doing things like analysing my profile or my social graph friends list instead of the approach popularized by Google Personalized Search where previous search queries are analyzed.

Of course, we live in an era when 10% of the Internet population doesn’t see anything wrong with brain implants to connect them to the Web so maybe I’m being paranoid when I worry that the next major leap in search engine relevance will only occur after we allow search engines to spy on us.

Now playing: Busta Rhymes - I Know What You Want (feat. Mariah Carey)

Before we got married, Jenna and I talked about pets and both agreed we didn’t want any. We didn’t want some animal that would urinate on the carpet, need to be taken outside for walks where I’d have to pick up it’s excrement, and that would create an additional inconvenience every time we decided to go out of town.

I’m sure you know where this is going…

We pick him up next week. Since I’m so against the idea, I get to name him. So far I’m torn between “Socks” and “Buster” both of which my wife thinks are horrible names for a dog. Any suggestions?

Now playing: Snoop Doggy Dogg - Doggy Dogg World

The folks at LiveSide have a blog post entitled Windows Live Spaces at a crossroads: will the US catch up to the world? which contains some interesting charts from ComScore. Specifically they call out the difference in the worldwide reach of various social networking services versus the North American reach. The relevant excerpt from the post is

As expected, MySpace is in a runaway lead, Facebook is coming on strong, Blogger is hanging in there, and Spaces pretty much brings up the rear.  If you read the blogs and follow Techmeme, TechCrunch, and Scoble, these numbers aren't anything surprising.

But take a look at the Worldwide numbers, and a somewhat different story emerges.

Social Networking Sites - Worldwide: Unique visitors per month (000)

While Facebook is growing steadily worldwide, here the numbers tell a far different story.  Windows Live Spaces is battling it out with Blogger and MySpace for the top spot.  Just for reference, we can see that the Worldwide usage of Social Networking sites is growing steadily:

You might quibble with the title of this blog post but it is hard to argue that Blogger is a social networking site by any definition of the term. When it comes to reach, no social networking site impacts as many users as Windows Live Spaces.

Of course, unique users aren’t the only metric Web sites are judged against and I’m sure there are many out there who will be quick to point out other charts that show our user engagement is lower than average which is a fair point. Personally, I suspect that the inclusion of the improved What’s New page will increase user engagement in a measurable way. It might just be me but I find myself visiting my What’s New page several times a day, in fact more often than I visit my Facebook news feed.

According to Facebook, the addition of the News Feed increased their page views by 70% in the first few months. I wonder if we’ll see a similar jump in the ComScore charts for Windows Live Spaces in a few months or whether FB’s results were an abberration. Only time will tell.

Now playing: Raekwon - Guillotine (Swordz) (feat. Ghostface Killah, Inspectah Deck & GZA/Genius)

Today I saw a pretty useless product announced on TechCrunch called FriendCSV which extracts your Facebook friends into a CSV file sans their PII (i.e. no email address, no IM screen names, no telephone numbers, no street address). Amusingly enough, the authors of the application brag about using the Facebook platform as it was designed to be used as if they have figured out some major exploit.

However it did remind me of a pretty cool desktop application that I found via Bubba called OutSync. It synchronizes the photos of your contacts in Outlook with those of any matching people found on your friends list in Facebook. If you have a Windows Mobile smartphone that is synchronized with your Exchange server like I do, it means that you get all your friends photos on your phone which is pretty sweet especially when you get a call from that person. Screenshots below

There’s that user-centric authentication model where the user grants the application access without giving the application their username/password. Again, I’m glad OAuth is standardizing this for the Web.

When I think of the Facebook platform embracing the “Web as a Platform” I want to see more applications like this enabled. Instead of only utilizing my Facebook social graph in the Facebook Marketplace or the Buy.com Facebook application, why can’t it be utilized on eBay or Craigslist? I want all the applications I use to be able to utilize my social graph to make themselves better without having to be widgets on a particular social networking site before they can take advantage of this knowledge. 

Back in 2004, I wrote

Basically, we've gotten rid of one of major complaints about online services; maintaining to many separate lists of people you know. One of the benefits of this is that you can utilize this master contact list across a number of scenarios outside of just one local application like an email application or an IM client. For example, in MSN Spaces we allow users to use their MSN Messenger allow list (people you've granted permission to contact you via IM) as a access control list for who can view your Space (blog, photo album, etc). There are a lot more interesting things you can do once the applications you use can tell "here are the people I know, these are the ones I trust, etc". We may not get there as an industry anytime soon but MSN users will be reaping the benefits of this integration in more and more ways as time progresses.

It’s unfortunate that almost 3 years later we haven’t made much progress on this across the industry although it looks like the Facebook platform has gotten people finally thinking about unified social graphs. Better late than never, I guess. 

The one mistake I’ve been making in my thinking has been narrowly defining the applications that should have access to your Windows Live social graph as Microsoft or Windows Live applications. My thinking has since evolved as has that of lots of folks in the B0rg cube. It will be an interesting couple of months with regards to social graph APIs especially with Google’s November 5th announcement coming up.

PS: Anyone else noticed that the installer for OutSync is hosted on SkyDrive?

PPS: I got a phone call from Rob Dolin earlier this evening and the pic from his Facebook profile showed up my phone. Sweet!

Now playing: Soulja Boy - Crank That (Soulja Boy)

For a while, I’ve been jealous of the Flickr API Explorer and the Facebook API Test console. So I started building one for the Windows Live Contacts API last week only to find out that there was already one out there which hadn’t been sufficiently publicized.

If you’re a developer interested in seeing what the Windows Live Contacts API offers and already have a Windows Live ID (formerly Passport account) then mosey on down to the the Live Data Interactive SDK page. Once you get there, click on the link that says Click to Request Permission , scroll down and hit [Give Access] to give the page access to your Windows Live address book. Once redirected, click “Work with Contacts” and then you can create, retrieve, update and delete people from your Hotmail and Messenger contact lists using a simple RESTful protocol with direct access to the XML responses. 

Below is a screenshot of me retrieving my Messenger contact using the interactive SDK.

Now playing: Baby Bash - Cyclone (feat. T-Pain)