The most interesting news from Facebook’s F8 last week was the announcement of App Links. If you are unfamiliar with the announcement, watch the 1 minute video embedded below which does a great job of setting up the sales pitch. Using App Links, mobile app developers can put markup in their web pages that indicate how to launch that page in their application on Android, iOS and Windows Phone. For example, clicking on a link to a FourSquare check-in from the news feed will launch the FourSquare app on your phone and will open that specific location or event.  .

The interesting question is why is Facebook doing this? It boils down to the fact that Facebook is an advertising company which makes the majority of its revenue from those ads asking you to install Candy Crush and Clash of Clans in your news feed.

Facebook’s pattern at this point is well known. They give you something of value for free (traffic) and once you get hooked they dial it down until you have to pay. The world is littered with the ashes of various companies who were once media darlings because Facebook gave them a bunch of free traffic from liberal news feed algorithms and then turned off the spigot. Just ask Viddy, all those social readers, Zynga, or read that hilarious break up letter from those guys at Eat24.

Publishers who use app links will likely get a boost in the news feed algorithm likely under the pretext that they provide a better user experience to consumers. Early success stories will cause lots of developers to create app links and then get hooked on the traffic they get from Facebook. Eventually your traffic will start dropping and any complaints will be met with an elaborate mathematical formula which explains why your content isn’t that hot on Facebook anymore. But don’t worry, you can fix all that by buying ads. Smile 

It’s obvious, devious and I love it. Especially since it does actually move the user experience of the mobile web forward even if the end goal is to make Facebook tons of money.

The other thing I give Facebook props for is holding a mirror up to the major search engines to see how silly we were being. Bing supports standards for app linking but it's only for Windows & Windows Phone apps. Google supports the same and again it only works for Android apps. Facebook is trying to say it doesn’t matter if you are on the web, Windows Phone, Android or iOS, links in the news feed should open in the native app on that platform. Google and Bing’s search engines on the other hand only supported the same when searching on the OSes from their parent companies. #strategytax

Hopefully Facebook’s move will bring more inclusiveness across the board from many online platform providers not just search engines. For example, I would love it if email providers also supported app links as well.

Note Now Playing: DJ Snake & Lil JonTurn Down For What Note


 

Danny Sullivan wrote an interesting blog post this morning titled Google’s Broken Promises & Who’s Running The Search Engine? whose central thesis is that Google now does a number of things it once described as “evil” when it comes to how search results and ads work in Google Search. Given that I now work in Bing Ads, this is a fairly interesting topic to me and one I now have some degree of industry knowledge about.

Promises Are Like Pie Crust, Easy to Make and Easy to Break

Danny Sullivan categorizes two big broken promises in his article, one in 2012 and one from 2013. The 2012 broken promise is excerpted below

The first broken promise came last year, when Google took the unprecedented step of turning one of its search products, Google Product Search, into a pure ad product called Google Shopping.

Google Shopping is a different creature. No one gets listed unless they pay. It’s as if the Wall Street Journal decided one day that it would only cover news stories if news makers paid for inclusion. No pay; no coverage. It’s not perfect metaphor. Paid inclusion doesn’t guarantee you’ll rank better or get favorable stories. But you don’t even get a chance to appear unless you shell out cold hard cash.

What Was Evil In 2004, Embraced In 2012

Shopping search engines have long had paid inclusion programs, but not Google. Google once felt so strongly that this was a bad practice that when it went public in 2004, it called paid inclusion evil, producing listings that would be of poor relevancy and biased. The company wrote, in part:

Because we do not charge merchants for inclusion in [Google Shopping], our users can browse product categories or conduct product searches with confidence that the results we provide are relevant and unbiased.

There is a similar Google then versus Google now perspective when looking at the second broken promise related to banner ads in the search results page.

“There will be no banner ads on the Google homepage or web search results pages,” Google promised in December 2005, on its main blog, to reassure consumers concerned that its new partnership with AOL would somehow change the service. Eight years later, Google’s testing big banner ads like these:

These excerpts could almost be a cautionary tale to idealistic young companies about making absolute statements and the staking one’s brand on these statements without thinking about the future. However that isn’t the point of this post.

I decided to write this post because Danny Sullivan’s article starts starts out quite strongly by pointing to this misalignment between Google’s past statements and their current behavior but then peters out. The rest of the article is spent studying Google’s org chart trying to figure out which individual to blame for these changes as well as trying to come up with a rationalization for these moves in the context of making search better for consumers. As an industry watcher the rationale for these moves is quite straightforward and has been a natural progression for years.

The Love of Money is the Root of all Evil

Any analysis of business decisions Google makes in the arena of search, is remiss if it fails to mention Google makes the majority of its revenue and profits from ads running on Google sites. As an example, Google made $9.39 billion last quarter from ads running on its sites whereas of the $3.15 billion it made from ads running on other people’s websites it paid those people $2.97 billion. Combining that with the fact that its $12.5 billion acquisition of Motorola has so far produced nothing but financial losses, there is a lot of pressure for Google to make as much money as possible from ads running on its sites specifically in Google Search results pages. 

When it comes to search engine advertising, the money is primarily in queries with “commercial intent”. This is a fancy way of saying that the person who is performing the search is planning to spend money. Advertisers are willing to pay several dollars to a search engine each time a customer clicks on their ad when the search term has commercial intent. In fact, companies are willing to spend up to $40 – $50 each time a user clicks on an ad if the user is searching for a life insurance policy or a home loan.

Over time both search engines and advertisers have figured out exactly where the money is and how to extract the most value from each other. Google has slowly been making changes to their search engine that implies that for queries with commercial intent they always want a cut of the action. This is why if you perform a search today that has commercial intent, there are an order of magnitude (i.e. ten times) as many links to ads as there are unpaid search engine results. For example, take a look at this screenshot of a query for “northface jackets” on Google.

There are two links on this page that are unpaid search results and eighteen links where Google gets paid if you click on them. Given that context, it is no surprise that Google eventually realized it was competing with itself by having a “free” shopping search engine. This explains the broken promise in 2012 related to paid inclusion.

Now if you take a close look at the above screenshot, you’ll notice that The North Face is actually the top advertiser on this page. This means that despite the fact that the user was specifically looking for the North Face brand products, the company has to still compete with other advertisers by paying people for clicks to their website from Google search results. Brand advertisers hate this. A lot. Not only did they spend a lot of money and effort to become a well-known brand but now they still end up paying when this brand recognition pays off and people explicitly are looking for them on Google.

This leads us to the second broken promise, banner ads in search results. What Google is trying to do is to appease brand advertisers by letting them “take over” the search engine results page in cases where the user is quite clearly searching for their brand. Treating this is a giant billboard that reinforces their brand as opposed to scrabbling with other advertisers for a user which they already consider theirs is a more amenable pitch. This explains the broken promise of 2013.

I expect to see more aggressive commercialization of the search results page given Google’s seeming lack of interest and inability to diversify their sources of income. Doing this while preserving the customer experience will be the #1 challenge of their search engine and other similarly advertising focused web products in 2014 and beyond.

Note Now Playing: Jay-ZF*ckwithmeyouknowigotit (featuring Rick Ross) Note


 

The big news in tech circles is this is the week Google Reader died. There has been a lot of collective shock about this since Google Reader was a staple of the sort of technology news omnivore and early adopter that was once considered to be the core fan base for Google. Despite Google’s protestations it’s quite clear that there was ample usage of the service with sites like Buzzfeed providing evidence that up to its last day it was driving over a million referrals a day. As someone who wrote a feed reader that synced with Google Reader which has been downloaded over 1.5 million times and is credited in Atom format specification it is sad to see such a core part of the RSS ecosystem come tumbling down.

The biggest surprise for many is how far Google seems to have strayed from its core mission as epitomized by this comment on Hacker News

ivank 1 day ago | link | parent

Organizing the World's Information... and setting it on fire 8 years later.

Like many companies before it, the company has done a great job of separating its public persona from the gritty reality of its business practices. Google’s business mission is to organize all the world’s information and use that to sell advertising. The great thing about this business model is that for a long time it has been a win-win for every one involved in the ecosystem. Consumers get great web services and products that are expensive to build and run for free such as Google Search, Gmail, Chrome and Android. Advertisers get to target consumers in new ways which have a better return on investment than they traditionally have been able to get from print and other media. While Google has minted hundreds of millionaires with the profits they’ve gotten from creating this ecosystem.

Unfortunately, Google has also hit the same problems that successful companies before it have had to face. The laws of large numbers mean that for Google to continue to be a great business for its shareholders (which includes its employees and executives) then it needs to find ways to eke even more money out of its customer base (advertisers) by giving them even more product (consumers with rich demographic and behavioral information) or by growing into new markets. Companies like Apple and Microsoft have faced this problem by growing new multi-billion dollar businesses. Apple went from a computer company to the iPod company and now it’s the iPhone company which is transitioning to becoming the iPad company. Microsoft started off as the BASIC company, transitioned to being the Windows & Office company and today has several billion dollar businesses from game consoles to database software. Google has decided to face this challenge by doubling down on their advertising business which means trying to suck in even more data via acquisitions like Waze and ITA as well as extracting more revenue from advertisers by removing consumer targeting options.

For the most part Google has diverted tech press scrutiny from this ongoing attempt to monopolize data sources and hover up personal information about users for later resale using misdirection. A few years ago the Google playbook was to release some technology branded as “open” such as OpenSocial which aimed to paint Facebook as “evil”, open sourced Android which paints Apple as “evil” or Google Chrome which was released with an excellent comic book explaining why it was so open. How could one complain about Google with a straight face when they were giving all of these free services to consumers and “open” technologies to the industry? No one would buy it. This was the strategy of the old Google under Eric Schmidt and it worked beautifully.

It has been an interesting hallmark of Google under Larry Page that it doesn’t play these games anymore. Pursuing Google’s business strategies at the expense of its public image as the good guy of the technology industry is now par for the course. I was taken aback when Google announced that it was going to fork Webkit since this was a blatant political move which many technology savvy geeks would see through. For days afterwards, the article A Short Translation from Bullshit to English of Selected Portions of the Google Chrome Blink Developer FAQ was a viral hit among developers on Twitter with excerpts such as

1.2 Is this new browser engine going to fragment the web platform's compatibility more?

Yes.

We intend to distract people from this obvious problem by continually implying that our as-yet unwritten replacement is somehow much better and more sophisticated than the rendering engine that until yesterday was more than good enough to permit us to achieve total dominance of the Windows desktop browsing market in less than two years.

This strategy has worked extremely well for Netscape, Microsoft, Apple and us in previous iterations of the browser wars, and we firmly believe that everyone in this industry was born yesterday and they will not recognise this for the total bullshit it so clearly is.

1.10 Is this going to be open source?

Not really.

While you can certainly read the source code, we're fully aware that actually tracking and understanding a modern HTML renderer is extremely difficult. In addition, the first changes we will make are intended specifically to break compatibility with WebKit, so the only organisation with sufficient resources to track our changes will no longer be able to do so.

In practice, this allows us to call the project "open" while simultaneously ensuring Google will be the only effective contributor to the Chrome and Blink source now and in the future. We've had enormous success co-opting the language of open source in the past to imply our products are better, and we aim to continue with that strategy.

So what does all of this have to do with Google Reader? As Marco Arment points out in his excellent post titled Lockdown, Google is in a battle with Facebook and Twitter to suck up user demographic and behavioral data around social signals. Google Reader is literally the opposite of that strategy since it is far too decentralized when compared to the model pioneered by Facebook Pages and Twitter.

Google Reader has been living on borrowed time since Facebook and Twitter became prominent. The only thing that has changed in 2013 is that Google’s management doesn’t think it’s worth it to throw a bone to millions of geeks and early adopters by keeping the lights running on the service with its existing skeleton crew. This new Google doesn’t care if geeks and early adopters just see it as another money hungry corporation that only focuses on the bottom line. Larry Page doesn’t give a shit.

Welcome to the new Google.

Note Now Playing: J.Cole - Can't Get Enough (featuring Trey Songz) Note


 

In a past life, I worked on the social news feed for a number of Microsoft products including the Messenger Social feed in Hotmail & Messenger and most recently the People app in Windows 8. When I worked on these products, we strongly believed in the integrity of the user experience and so never considered the social feed as a canvas for showing users ads.

Thus I read a pair of recent posts by Dalton Caldwell, founder of App.net, with some interest. Dalton wrote about the recent moves that both Twitter and Facebook are making to ensure that the social feeds on their sites become a great canvas for showing ads.

In his post Understanding Like-gate Dalton writes

The best ad is indistinguishable from content

We can expect to see Facebook deemphasizing traditional advertising units in favor of promoted news stories in your stream. The reason is that the very best advertising is content. Blurring the lines between advertising and content is one of the most ambitious goals a marketer could have.

Bringing earnings expectations into this, the key to Facebook “fixing” their mobile advertising problem is not to create a new ad-unit that performs better on mobile. Rather, it is for them to sell the placement of stories in the omnipresent single column newsfeed. If they are able to nail end-to-end promoted stories system, then their current monetization issues on mobile disappear.

In his post Twitter is pivoting Dalton writes

Predicting the future

In this paradigm, Twitter’s business model is to help brands “amplify their reach”. A brand participating in Twitter can certainly distribute their content for free and get free organic traffic, but if they want to increase their reach, they need to pay.

It’s no accident that this sounds exactly like the emerging Facebook business model. As discussed in that link, algorithmically filtered primary feeds are vastly easier to advertise against vs unfiltered feeds. The issue for Twitter is that Facebook already has a far larger userbase which is already trained to read an algorithmically filtered feed.

In a twist, I wouldn’t have predicted a few years ago it is now a regular occurrence for both users of Facebook and Twitter to see ads in their feeds. Twitter has gone as far as effectively crippling new Twitter apps to ensure that every Twitter user gets an ads-heavy unfiltered Twitter experience. The reason for this is straightforward. Both companies have sky high expectations from investors as evidenced by Facebook's $100 billion valuation it has failed to meet and Twitter's $8 - $10 billion valuation on $100 million in revenues. The challenge for both services is that investors are expecting Google-like returns on investment but neither of these companies have a Google-like business model.

The problem with ad supported online businesses is that for the most part their business models suck. In a traditional business, if you focus on building a great product or service that provides an excellent customer experience then you will be making money hand over fist. In most ad supported online businesses, your business is selling your product’s audience as opposed to the product itself. That means if you want to make more money you have to pimp out your audience often in disrespectful and intrusive ways to eke out that extra dollar.

The one place where this is different is online search (i.e. Google’s primary business). In the web search, the ads aren’t just indistinguishable from content but in the most lucrative cases, the ads are better than the content. As an example take a look at these searches

Since we may get different results, I’m including a screenshot below

There are 8 ads in this screenshot and 2 search results. However instead of being irritated as I would be if the ratio of ads to content was 4:1 in a YouTube video or Facebook feed, the ads are actually more relevant than the organic search results. This is the holy grail that Twitter and Facebook are trying to achieve.

As Dalton points out, Facebook has already socialized its users to the notion that brands will post commercial messages to the feed. In addition, brands have grown so entitled to it then when asked to pay for them since they are ads, they get outraged. However Facebook has been boiling this particular frog for a while. Facebook encourages advertisers to create Pages and users to like Pages so that they can stay connected to the brands they care about. Content in your feed from people and brands you don’t follow snuck in under the aegis of showing you content your friends interacted with. Finally not only has Facebook had promoted posts for brands for a while, they now also allow users to promote their personal posts to friends for $7 a pop.

Without really thinking about it much, we’re halfway to a future where a significant percentage of the content of your Facebook feed is paid. Since the posts go through the same ranking algorithm as your regular feed, they are more likely to be relevant to you than the traditional ad products that Facebook and other online properties are known for today. When the goal is to be entertained, do you really care if that viral video of the day being shared via a friend is a paid impression or not? 

Twitter is playing catch up here but if they don’t, the flop that was Facebook’s IPO will look tame in comparison.

Note Now Playing: Macklemore & Ryan Lewis - Can't Hold Us Feat. Ray Dalton Note


 

July 10, 2011
@ 01:02 PM

I’ve been joking with Omar that Google+ is the new FriendFeed. I recently posted this on Google+ and was asked to explain what I meant since Google+ doesn’t support importing of content from other services which was the key feature FriendFeed. The reason I say this is that Google+ fulfills the same need that FriendFeed when it first came out.

Here’s an excerpt from a post by Robert Scoble in 2008 about FriendFeed titled Loving my FriendFeed

I love my FriendFeed. Here’s a list of top bloggers who are using the service. Why do I love it? It’s one place you can find all my stuff and, even, comment on it. It’s amazing the discussions that a 140-character “Tweet” on Twitter can generate. I subscribe to a ton of people on FriendFeed and notice that often the conversations after a Twitter message will be 1000x longer (and generally more interesting) than the Twitter itself.

In my previous post I asked what problem Google+ solves and the answer is above. Google+, like FriendFeed before it, gives people a place to subscribe to and participate in conversations around content produced by people they are interested in.

Why Twitter Doesn’t Solve This Problem

Twitter relationships have been described as a public interest graph. Specifically, Twitter is a way to keep on top of people and content you find interesting whether it is tech news sites, bloggers, celebrities, government officials and even people you know. However there are a number of key gaps in the Twitter user experience which FriendFeed fixed and Twitter still hasn’t even though people have been complaining about them for years.

The first problem is that is really difficult to have conversations on Twitter. Here’s an excerpt from a TechCrunch post made in 2008 titled Actual Conversations On Twitter Not Possible Until Twitter Lets Us which explains the problem

One of the big complaints about Twitter is that conversations are hard to follow. Users can write a response to a Twitter message (or anything else), but the easy way to do this is to add an @[username] tag to the Twitter, which refers back to the original Twitter user. But by then that original user has often moved on to other subjects, and it becomes impossible to follow the conversation.

The fact is that Twitter purposefully doesn’t want users to be able to track conversations. The content begins and ends with a discreet Twitter message, up to 140 characters long. Competitor Friendfeed does a nice job of tracking conversations by letting users reply to actual messages, not just users. Twitter, for whatever reason (possibly to keep things simple), just doesn’t want that. And until they do, nothing is going to change.

The ability to have actual comment threads about a status update as opposed to disconnected @replies is a more satisfying experience for many users. As Mike Arrington stated above, the challenge for Twitter is that this would change the dynamics of the service in ways that take away some of the character of the service.

The second problem is that Twitter doesn’t give a public way to indicate that a piece of content is interesting without also sharing it. Specifically, there is no analog to Facebook’s “I like this” within the stream (not to be confused with the like button social plugin). Twitter has favorites but it’s actually meant to be a way to bookmark posts not to tell people you like the status update. There are now sites like Favstar.fm which have garnered a sizable user base by giving people a way to get “I like this” style functionality from Twitter and see how many people have favorited a tweet.

Both of these problems are fixed by Google+ and it is unsurprising that the same sorts of people who loved FriendFeed are not only on Google+ but are its most popular users. The question is whether Twitter will fix these problems with their experience given that this has made people pine for alternate services. Given that they didn’t try to address these when FriendFeed was at the height of its hype curve, it seems unlikely they will unless they see declines in their more mainstream user base.

Why Facebook Doesn’t Solve This Problem

Facebook relationships are an attempt to mirror our offline relationships online. The problem with this is captured in Paul Adams’ excellent slideshow The Real Life Social Network v2

The problem with Facebook is that people you may find interesting (i.e. your interest graph) or that find you interesting are not necessarily people you want to sharing the same space as your family, friends and even coworkers. A good example of this problem are the following suggestions I saw when I logged into Facebook this morning.

Alexia Tsotsis and Steven Levy are both journalists who work for TechCrunch and Wired respectively. Although I find the articles they write interesting, I don’t want to have them be on the receiving end my mobile phone videos of my son playing in the park or my check-ins from places around Seattle nor do I want to be subjected to their similar personal updates.

The combination of asymmetric following (people can subscribe to my updates without my accepting a friend request) and the ability to place people into groups (i.e. Circles) which can then be used to provide limited visibility to various updates is how Google+ solves this problem for various interest graphs. Neither of these features exists in Facebook today and while I suspect they will add the latter especially since Paul Adams now works there, it is harder to imagine seeing asymmetric follow ever showing up on Facebook outside of Pages.

Where That Leaves Us

I expect that both Twitter and Facebook will lose some chunk of people’s time to Google+. However Twitter is more vulnerable than Facebook, because Facebook has been fairly resistant the rise of the “interest graph” by building features like Facebook Pages which allows people to follow their interests in the same stream as updates from people they care about offline. For example, it is interesting to note that the most popular user on Twitter is Lady Gaga with 11.5 million followers but on the other hand her Facebook fan page has 40 million fans. Secondly, there really isn’t a gap Google+ fills with regards to communicating and staying in touch with the people one cares about offline via a social network.

On the other hand, Google+ is more in the same product space as Twitter being interest graph related which can be seen by the usage patterns of its early adopters. It’s also telling to read comments from Google+ readers on how much less time they now spend on Facebook and Twitter.

Note Now Playing: Frank Ocean - NovacaneNote


 

Two months ago Nelson Minar wrote a post entitled Stop making social networks, Facebook won where he argues that websites should just treat Facebook as the one true social graph instead of trying to build their own. I agree a lot with what Nelson wrote which some people tell me conflicts with my argument that There will be many social graphs. I thought that the best way to illustrate this seeming contradictory thinking is by comparing two sites that the media considers competitors to Facebook in different ways; Twitter and FourSquare.

How Twitter and FourSquare position themselves against Facebook

Recently, Twitter’s VP of business and corporate development spoke at Nokia World 2010 where he proclaimed that Twitter is NOT a social network. Below are some excerpts from ReadWriteWeb’s coverage of his talk.

says Thau: Twitter is for news. Twitter is for content. Twitter is for information.

So Twitter Is "News"?

Yes, says Thau. Twitter is changing the very nature of news today. Journalists are sending their stories to Twitter and some are even publishing directly to Twitter. It's also allowing everyday users to become journalists themselves by providing them with a simple mechanism to break news.

"The guy who saw a plane land on the Hudson River right in front of him didn't think to send an email," says Thau. "He tweeted it."

Thau also wanted to assure Twitter users it's OK if you think you're not interesting enough to have your own Twitter account. Don't apologize if you don't tweet - just come to Twitter and consume content instead. After all, plenty of people already do just that.

The key thing here is that Twitter is arguing that the primary relationships on the site are not “social”. They are producer<->consumer where the product is news and other information content.

Dennis Crowley of FourSquare made more direct comparisons between his service and Facebook in an interview with TechCrunch.

On why the world needs more than one social graph

Our social graph is more representative of the people that you meet in the real world. I am starting to believe, if you asked me a year ago, Why would you ever need more than one social graph? You need representation of a couple of them. Between the three, Facebook is literally everyone I’ve ever shaken hands with at a conference or kissed on the cheek at Easter. Twitter seems to be everyone I am entertained by or I wish to meet some day. Foursquare seems to be everyone I run into on a regular basis. All three of those social graphs are powerful in their own.

The FourSquare argument is that services that create new social graphs that are tied to a specific social context can continue to exist and grow in a world where social networking is dominated by Facebook’s website and Facebook Connect.

 

Facebook’s trajectory: Adding a social element to every online activity

Before analyzing the wisdom of the approaches that FourSquare and Twitter have taken to differentiate their offerings from Facebook, it is a good idea to have an idea of Facebook’s ultimate strategy. This isn’t hard since Zuckerberg and other Facebook regularly share this with TechCrunch. Below is an excerpt from an article titled Zuckerberg: Facebook Photos Used 5 Or 6 Times More Than Competitors — Combined which describes their long term strategy

He noted that when they launched the product, they didn’t have all of the features that their competitors did. For example, they didn’t have high-resolution photos and you couldn’t print them. But one thing they did have was the social element — and this changed everything.

“Those features by themselves were more important than anything else combined,” Zuckerberg said of the social elements of Facebook Photos. He then dropped the competitor bomb. “The photo product that we have is maybe five or six times more used than every other product on the web — combined,” Zuckerberg stated.

And it was clear from both Zuckerberg and CTO Bret Taylor’s talk at the event that photos to them was the harbinger of things that eventually came — and will still come.

Taylor noted that he had been “brainwashed by Silicon Valley” before he saw and understood the power of Facebook Photos (he was likely working at Google at the time). He had been thinking like an engineer about the best way to organize photos on the web. But he quickly realized that “the best possible organization of photos is around people,” Taylor said.

“There are ten other industries waiting to have this type of disruption,” Taylor said noting the travel industry, e-commerce, and music as a few of them. Earlier, Zuckerberg agreed. Because of the social element, “every single vertical will be transformed.“

Facebook’s social graph is a graph of people I know or have met. Facebook’s fundamental strategy is to build a product and platform where key online activities are improved by adding the social element of people you know. Where Facebook has been dominant is when the activity is one that already related to interacting with people you know. Facebook has beaten geek favorites like Gmail, Flickr and Delicious as the way regular people share private messages, photos and links online. This is both due to the powerful network effects of a social networking product and the fact that their graph maps 100% to the people one typically wants to indulge in those activities with.

Facebook has had less success with products where their graph doesn’t correspond well with the target activity. The best examples of this are Facebook Marketplace versus eBay/Craig's List or Facebook Questions versus Yahoo! Answers. In both of these comparisons, Facebook isn’t even on the radar of the market leader. This is because activities such as buying someone’s old junk really need a wider net than just your friends, family and coworkers.

This is where the positioning and focus of Twitter as a news service as opposed to a social network puts them in a good place in comparison to Facebook. Twitter is where I go to get entertainment and news about the topics I’m interested in from subject matter experts. These subject matter experts (in many cases bloggers, minor & major celebrities) are not people I know nor  have I met. This is distinct from my Facebook social graph but has some overlap depending on how much of a subject matter expert I am myself. On the other hand, FourSquare is a place where I go to share my location with people I know or have met. This set of people is almost always a subset of the people in my Facebook social graph. The only value additions you get from FourSquare are the game mechanics and deals (not anymore). FourSquare has unfortunately reached the point where the only practical difference between using it versus Facebook Places is that I get to be mayor of my local Gymboree and collect two dozen video game style achievements. Personally I’ve already grown bored with the game mechanics and suspect that targeting the console gaming demographic guarantees it will be a niche service at best.

The bottom line is that if the primary focus of your product is that it connects people with their friends, family and others they know around a particular activity then you need to be able to answer the question as to how your product can compete in a world where your service is a feature of Facebook or of an app on its platform.

Note Now Playing: Far East Movement - Like a G6 Note


 

July 15, 2010
@ 02:20 PM

I was reading Pandas and Lobsters: Why Google Cannot Build Social Applications... and came across the following statements

Now, consider the Four Horsemen of Hotness in 2010: Facebook, Quora, Foursquare, and Twitter. Think deeply about why none of these four could have been developed inside Google.
...
Quora is restaurant that serves huge quantities of
bacn and toast. Quora is a dozen people running dozens of experiments in how to optimally use bacn to get people to return to Quora, and how to use toast to keep them there. Bacn is email you want but not right now, and Quora has 40 flavors of it that you can order. Quora's main use of Bacn is to sizzle with something delicious (a new answer to a question you follow, a new Facebook friend has been caught in the Quora lobster trap, etc.) to entice you to come back to Quora. Then, once you're there, the toast starts popping. Quora shifts the content to things you care about and hides things you don't care about in real-time, and subtly pops up notifications while you're playing, to entice you to keep sticking around and clicking around.

Although I’m a regular user of Foursquare, Twitter and Facebook and consider myself to be fairly up on what’s going on in the social media space, I’d never used Quora when I read the article. Given that I’ve seen hype about it in various corners I decided to create an account and give the service a try. Below are some of my impressions

What is Quora?

The easiest way to think about Quora is that Quora is to Yahoo! Answers as Facebook is to MySpace. It is a Q&A site where users utilize their real names often linked to Facebook profiles as opposed to pseudonyms. It avoids game mechanics such as high score leaderboards and user badges that services like Stackoverflow and Windows Live QnA instead relying on people getting kudos from their peers in the form of endorsements and votes on their answers to motivate users to answer questions.

Why is Quora is so Hot

Quora seems to have started off as an invitation-only service which allowed them to cherry pick the original users to meet a particular demographic (i.e. Silicon Valley geek) and also carefully manage the initial culture of the site. What they’ve created is a place where members of the Silicon Valley technorati and wannabes can post questions and expect them to be answered by members of the tech elite including insiders at various tech companies. Quora is the kind of place where questions like What are the scaling issues to keep in mind while developing a social network feed? is answered by one of the people who built the original Facebook news feed and a random opinion like Should Mark Zuckerberg step down as CEO of Facebook and find a more seasoned replacement? gets a response from Blake Ross. There aren’t many places online where a question like What were the 4 or 5 key decisions that Larry Page and Sergey Brin made in the early days of Google? can get a serious answer let alone a well researched history lesson as well as some actual insights.

This site is pure gold for technology bloggers, journalists and Web startup geeks. It is unsurprising that these sorts of people would consider Quora to be the greatest thing since sliced bread.

Quora is a Community Site not a Communications Tool

One of the things I find weird about lumping Quora into the same grouping as Facebook, Foursquare and Twitter is that unlike those sites it is not a communication tool. Facebook created a new communication channel between friends, acquaintances and family members that sits somewhere between brings together the functionality of email and IM along with the feed. Twitter created a lighter weight way to consume and create content for brands and people you find interesting as compared to blogging. Foursquare is about broadcasting your location to interested parties.

Quora on the other hand seems to have more in common with mail lists and discussion forums. Specifically, it is more like Metafilter, Digg and Reddit than it is like the aforementioned sites. This is a service that will live and die based on the culture of its community and is very dependent on "power users” who altruistically provide lots of value to the site in exchange for respect from their peers. The challenge for Quora is that it will be difficult to keep its current culture as it grows bigger. Will Facebook and Google insiders still be showing up in various question threads if the site grows to be as big as Yahoo! Answers with the same breadth of audience and volume of content? I can’t imagine that happening.

I also can’t imagine being able to segregate audiences like you can on communications services. Twitter has communities of mommy bloggers, tech bloggers, fans of various celebrities, sports fans, etc which operate independently of each other and really only are noticed by others every once in a while due to the trending topics feature. The same goes for Facebook and Foursquare. Quora will not be able isolate the various demographics from each other without changing the nature of the site. However they will have to figure that out once the current crop of users start logging in and seeing "how is babby formed" style questions because the site has taken off.

Then again, we might get lucky and the site never take off with the masses which may not be good for the VCs that have invested in it but would be for the community that has formed there.

Note Now Playing: Young Buck - Welcome to the South (featuring Lil Flip & David Banner) Note


 

In a move that was telegraphed by Fred Wilson’s (Twitter investor) post titled The Twitter Platform's Inflection Point where he criticized Twitter platform developers for “filling holes” in Twitter’s user experience, the Twitter team have indicated they will start providing official Twitter clients for various mobile platforms. There have been announcements of an official Blackberry client and the purchase of Tweetie so it can become the official iPhone client. The latter was announced in the blog post Twitter for iPhone excerpted below

Careful analysis of the Twitter user experience in the iTunes AppStore revealed massive room for improvement. People are looking for an app from Twitter, and they're not finding one. So, they get confused and give up. It's important that we optimize for user benefit and create an awesome experience.

We're thrilled to announce that we've entered into an agreement with Atebits (aka Loren Brichter) to acquire Tweetie, a leading iPhone Twitter client.

This has led to some anger on the part of Twitter client developers with some of the more colorful reactions being the creation of the Twitter Destroyed My Market Segment T-shirt and a somewhat off-color image that is making the rounds as representative of what Twitter means by “filling holes”.

As an end user and someone who works on web platforms, none of this is really surprising. Geeks consider having to wade through half a dozen Twitter clients before finding one that works for them a feature even though paradox of choice means that most people are actually happier with less choices not more. This is made worse by the fact that in the mobile world, this may mean paying for multiple apps until you find one that you’re happy with.

Any web company that cares about their customers will want to ensure that their experience is as simple and as pleasant as possible. Trusting your primary mobile experience to the generosity and talents of 3rd party developers means you are not responsible for the primary way many people will access your service. This loss of control isn’t great especially when the design direction you want to take your service in may not line up with what developers are doing in their apps. Then there’s the fact that forcing your users to make purchasing decisions before they can use your site conveniently on their phone isn’t a great first time experience either. 

I expect mobile clients are just the beginning. There are lots of flaws in the Twitter user experience that are due to Twitter’s reliance on “hole fillers” that I expect they’ll start to fill. The fact that I ever have to go to http://bit.ly as part of my Twitter workflow is a bug. URL shorteners really have no reason to exist in the majority of use cases except when Twitter is sending an SMS message. Sites that exist simply as image hosting services for Twitter like Twitpic and YFrog also seem extremely superflous especially when you consider that since only power users know about them not every Twitter user is figuring out how to use the service for image sharing. I expect this will eventually become a native feature of Twitter as well. Once Twitter controls the primary mobile clients for accessing their service, it’ll actually be easier for them to make these changes since they don’t have to worry about whether 3rd party apps will support Twitter image hosting vs. Twitpic versus rolling their own ghetto solution.

The situation is made particularly tough for 3rd party developers due to Twitter’s lack of a business model as Chris Dixon points out in his post Twitter and third-party Twitter developers

Normally, when third parties try to predict whether their products will be subsumed by a platform, the question boils down to whether their products will be strategic to the platform. When the platform has an established business model, this analysis is fairly straightforward (for example, here is my strategic analysis of Google’s platform).  If you make games for the iPhone, you are pretty certain Apple will take their 30% cut and leave you alone. Similarly, if you are a content website relying on SEO and Google Adsense you can be pretty confident Google will leave you alone. Until Twitter has a successful business model, they can’t have a consistent strategy and third parties should expect erratic behavior and even complete and sudden shifts in strategy.

So what might Twitter’s business model eventually be?  I expect that Twitter search will monetize poorly because most searches on Twitter don’t have purchasing intent.  Twitter’s move into mobile clients and hints about a more engaging website suggest they may be trying to mimic Facebook’s display ad model.

The hard question then is what opportunities will be left for developers on Twitter’s platform once the low hanging fruit has been picked by the company. Here I agree with frequent comments by Dave Winer and Robert Scoble, that there needs to be more metadata attached to tweets so that different data aggregation and search scenarios can be built which satisfy thousands of niches. I especially like what Dave Winer wrote in his post How Twitter can kill the Twitter-killers where he stated

Suppose Twitter wants to make their offering much more competitive and at the same time much more attractive to developers. Sure, as Fred Wilson telegraphed, some developers are going to get rolled over, esp those who camped out on the natural evolutionary path of the platform vendor. But there are lots of things Twitter Corp can do to create more opportunities for developers, ones that expand the scope of the platform and make it possible for a thousand flowers to bloom, a thousand valuable non-trivial flowers. Permalink to this paragraph

The largest single thing Twitter could do is open tweet-level metadata. If I want to write an app for dogs who tweet, let me add a "field" to a tweet called isDog, a boolean, that tells me that the author of the tweet is a dog. That way the dog food company who has a Twitter presence can learn that the tweet is from a dog, from the guy who's developing a special Twitter client just for dogs, even though Twitter itself has no knowledge of the special needs of dogs. We can also add a field for breed and age (in dog years of course). Coat type. Toy preference. A link to his or her owner. Are there children in the household?

I probably wouldn’t have used the tweeting dog example but the idea is sound. Location is an example of metadata that is added to tweets which can be used for interesting applications on top of the core news feed experience as shown by Twittervision and Bing's Twitter Maps. I think there’s an opportunity to build interesting things in this space especially if developers can invent new types of metadata without relying on Twitter to first bless new fields like they’ve had to do with location (although their current implementation is still inadequate in my opinion).

Over the next few months, Twitter will likely continue to encroach on territory which was once assumed to belong to 3rd party developers. The question is whether Twitter will replace these opportunities they’ve taken away with new opportunities or instead if they’ve simply used developers as a means to an end and now they are no longer useful?

Note Now Playing: Notorious B.I.G. - One More Chance Note


 

February 15, 2010
@ 02:59 PM

From the Google Wave Federation architecture white paper 

Google Wave is a new communication and collaboration platform based on hosted documents (called waves) supporting concurrent modifications and low-latency updates. This platform enables people to communicate and work together in new, convenient and effective ways. We will offer these benefits to users of wave.google.com and we also want to share them with everyone else by making waves an open platform that everybody can share. We welcome others to run wave servers and become wave providers, for themselves or as services for their users, and to "federate" waves, that is, to share waves with each other and with wave.google.com. In this way users from different wave providers can communicate and collaborate using shared waves. We are introducing the Google Wave Federation Protocol for federating waves between wave providers on the Internet.

From a Buzz post by Dewitt Clinton, a Google employee

The best way to get a sense of where the Buzz API is heading is to take a look at http://code.google.com/apis/buzz/. You'll notice that the "coming soon" section mentions a ton of protocols—Activity Streams, Atom, AtomPub, MediaRSS, WebFinger, PubSubHubbub, Salmon, OAuth, XFN, etc.

What it doesn't talk much about is Google. That's because the goal isn't Google specific at all. The idea is that someday, any host on the web should be able to implement these open protocols and send messages back and forth in real time with users from any network, without any one company in the middle. The web contains the social graph, the protocols are standard web protocols, the messages can contain whatever crazy stuff people think to put in them. Google Buzz will be just another node (a very good node, I hope) among many peers. Users of any two systems should be able to send updates back and forth, federate comments, share photos, send @replies, etc., without needing Google in the middle and without using a Google-specific protocol or format.

From Mark Sigal’s post Google Buzz: Is it Project, Product or Platform?

I think that it's great that Google is iterating Gmail (read Tim O'Reilly's excellent write-up on it here), and actually improving an existing product, versus rolling out a knock-off of something that is already in the market.

Nonetheless. I am confused. I thought that Google Wave was destined to be the new Gmail, but after yesterday's announcement, I am left wondering if Gmail is, instead, the new Google Wave.


Since the saying goes that people in glass houses shouldn’t throw stones, I won’t make any comment besides sharing these links with you.

Note Now Playing: 50 Cent - Crime Wave Note


 

The New York Times has an article titled How Google Decides to Pull the Plug which talks about the rationale behind the rash of abandoned and cancelled projects that have come out of the Big G in recent months. The article contains the following interesting excerpts

GOOGLE recently set the blogosphere abuzz by announcing that it was pulling the plug on several products.

The victims included Lively, a virtual world that was Google’s answer to Second Life; Dodgeball, a cellphone service aimed at young bar-hoppers who wanted to let their friends know where they were hanging out; Catalog Search, which scanned paper product catalogs so they could be searched online; and Notebook, a simple tool that allowed people to take notes on Web sites they had visited.

Google also said it would stop actively developing Jaiku, a microblogging service similar to Twitter, and instead turn it over to its users as an open-source project they could tinker with as they wished.

All of the shuttered projects failed several of Google’s key tests for continued incubation: They were not especially popular with customers; they had difficulty attracting Google employees to develop them; they didn’t solve a big enough problem; or they failed to achieve internal performance targets known as “objectives and key results.”

You’d think that Google, a highly profitable engineer’s playground, would keep supporting quirky side projects as long as someone wanted to work on them. The company, which is best known to consumers for its search engine, is famous in business for promoting innovation by letting engineers devote 20 percent of their time to projects outside their main responsibilities.

But in this difficult economy, even Google is paying more attention to costs.

Lively, Google’s entry into three-dimensional virtual worlds, was publicly unveiled last July. Four months later, when the company decided to close it, only 10,000 people had logged into the service over the previous seven days. That was well below the targets set by Google’s quarterly project review process, and far behind Second Life from Linden Lab, which had about half a million users in a similar period.

“We didn’t see that passionate hockey-stick growth in the user base,” said Bradley Horowitz, Google’s vice president for product management. Management decided that the half-dozen people working on Lively could be more productive elsewhere.

It will be interesting to see what the long term effects of these changes in perspective will be on Google's culture around launching new products out of 20% time projects and the veneration of side projects at the Googleplex. One expected change is that employees will be more conservative around what 20% projects they choose to work on so that they don't end up wasting their time on the next Google Page Creator or Google Web Accelerator which is received with initial hype but quickly discontinued because it doesn't see "hockey stick growth in user base".

You can already see this happening somewhat by looking at how many side projects are being shipped as part of Gmail labs. Checking out the list of Gmail labs offerings I see over 30 big and small features that have been added to Gmail as side projects from various individuals and teams at Google. It seems on the surface that a lot of Google employees are betting on tying their side projects to a huge, successful product like Gmail which isn't in danger of being cancelled instead of working on new projects or helping out smaller projects like Dodgeball and Jaiku which eventually got cancelled due to lack of interest.

This expectation that a new Google product will need massive adoption to justify its investment or be cancelled within four months, as was the case with Google Lively, will be a significant dampener new product launches. Reading Paul Buchheit's post on the early days of Gmail I wonder how much time he'd have invested in the project if he was told that Google would cancel the project if it's user base growth wasn't competitive with market leaders like Yahoo! Mail and Hotmail's within four months.

I suspect that the part of Google's DNA which spurs innovation from within is being killed. Then again when you look at Google's long list of acquisitions you realize that a lot of their most successful projects outside of search including Google Maps, Blogger and YouTube were the results of acquisitions. So maybe this culture of internal innovation was already on the way out and the current economic downturn has merely sealed its fate.

Note Now Playing: Metallica - Fade To Black Note


 

Facebook made two interesting announcements this week in the area of making their platform more open. The first announcement was made by Mike Shroepfer in the developer blog post Next Steps in Openness where he writes

Enabling social information to flow through the Web is one of the core goals of Facebook. In the two months since Facebook Connect became generally available, over 4,000 sites and desktop applications have gone live with the service. Users can now log into sites across the Web using their Facebook account, bring their identity and friends with them, and share information and experiences using the same features as they would with applications on Facebook.

As we've launched and built Facebook Connect, we've been participants in OpenID efforts. One of our user experience experts, Julie Zhuo, presented at the UX Summit in October. Several of our engineers have been participating in meetups, and one of them ran as a community member for a board seat. We're happy to announce today that we are formalizing our support of the OpenID Foundation by officially joining the board.

At first glance it seems contradictory for Facebook to join the OpenID foundation given that many people view Facebook Connect to be a competitor to OpenID as a single sign-on solution albeit a centralized one. The confusion from pundits around this announcement from Facebook is summarized in the Ars Technica article Facebook's "next steps in openness" raises questions which openly questions why the company needs to join the OpenID foundation when Facebook Connect seems to be enjoying broader deployment across the Web.

The reason seems quite straightforward to me. Right now, Facebook has created a situation where web sites that want to enable their users log-in with credentials from third party sites need to implement two solutions; an OpenID consumer and Facebook Connect. This places a burden on web sites which now either have to chose to adopt one solution over the other or create a user experience problem by offering users both options. It's bad enough that there is now a market for companies who wrap both solutions in a single package like JanRain and Clickpass. I wouldn't be surprised if Facebook has been getting an earful from some of their larger Facebook Connect partners about being a competing offering to OpenID.

That said, I doubt that a generic solution like OpenID will ever beat the ease of use and value added functionality that Facebook has created with Facebook Connect. By supporting OpenID, Facebook gets to silence the sites who simply want single sign-on solution that is built on standards so they can implement one solution to get access to users of Facebook, Google, Yahoo!, Windows Live, MySpace, and AOL's services among others. However anyone who wants the richer integration that comes with integrating with the Facebook mini-feed and live feed will still need to adopt the proprietary Facebook Connect.

The second interesting announcement about Facebook opening up comes from Chris Putnam in the post titled Opening Up Facebook Status, Notes, Links, and Video to Facebook Platform which is excerpted below

We've seen increasing engagement with over 15 million users updating their status each day and sharing over 24 million links per month. We wanted to make sure this content and the ability to share this content was available through our standard APIs.

Specifically, your applications can now directly access all of a user's status, links, and notes via new methods and FQL calls. Your application will have access to any status, notes, or links from the active user or their friends that are currently visible to the active user. In addition, we're opening new APIs for you to post links, create notes, or upload videos for the current user, and we've made setting a user's status easier.

We're pretty excited to see what kinds of ideas you can come up with to help users create and share more content.

Like Facebook Connect, this is a further step away from the walled garden approach that characterized the Facebook platform of 2007. In 2007, the goal of Facebook's platform efforts was to tell websites to re-implement themselves as Facebook apps (e.g. iLike website vs. iLike Facebook app). With Facebook Connect and the changes to their APIs announced above, Facebook is now acknowledging that their users use other websites and integration with those sites doesn't require these sites to become sharecroppers on the digital farmland that is the Facebook canvas page.

Twitter has shown the value of having open APIs for link sharing and status updates which isn't lost on Facebook given their failed attempt to acquire Twitter. Sharing status updates is a powerful way for humans to connect in social contexts and given Facebook's goal to be the world's social operating system we can expect them to make more moves in this space. It's particularly interesting to see how people use status updates as a way to communicate and share experiences during news events such as President Obama's inauguration. Facebook saw this first hand with their integration of Facebook's status updates with the CNN website where Randi Zuckerburg wrote

Over 2 million status updates were posted through the CNN.com Live Facebook feed with 4,000 status updates per minute on average and a huge spike of 8,500 updates when President Obama began his speech. We're thrilled that you and so many others chose to actively participate as history was being made—by sharing the experience with the people that matter most to you.

Similar surges were seen on Twitter as evidenced by the chart below. The thing that is interesting to note is that while status updates increased during the most exciting part of the inauguration other forms of Web traffic declined at the same time.

Similar traffic drop on Last.FM seen on the right

Traffic Drop to Last.FM during Obama inauguration on right

Google saw a similar drop as well

Traffic Drop to Google during Obama Inauguration

Was it because everybody went to Twitter?

Traffic Spike on Twitter during Obama Inauguration

Charts taken from the post Understanding Web Operations Culture - the Graph & Data Obsession on the O'Reilly Rader blog.

UPDATE: I've been playing around with the APIs announced and it looks like there actually isn't an API to get your friends' status updates included in the announcement. The status.get method only returns the status updates posted by the currently user. The closest I could get was trying the query below via fql.query but it never returned results

SELECT uid, message FROM status WHERE uid IN (SELECT uid2 FROM friend WHERE uid1=$userid)

So much for the claims that this is a Twitter killer.

Note Now Playing: Eminem, Dr. Dre & 50 Cent - Crack a Bottle Note


 

This morning I was reading Om Malik's post entitled Google & The Big Ideas where he writes

Earlier today, when I read about Google launching a mobile version of Tasks. I was amazed by the attention being focused on essentially a to-do list website. And while it wasn’t worth a story, I shared my feelings via I sent Twitter. My tweet read:

I think google has no big ideas. this morning they announced a to-do-list. FGS. [For God Sake] Remember the Milk MUCH better.

...
Matt’s comments and the responses both on this blog and FriendFeed resulted in some thoughts about what constitutes a big idea, where Google is right and where it is light. Instead of responding on
FriendFeed, I decided to share my thoughts with you, hoping that we can have a larger conversation about Google and the big ideas.

For me startups and products such as Skype, Flickr and YouTube represent big ideas. Why? Because they not only redefine our notions about certain technologies, but they also change our behavior and cause massive disruption. For instance, Skype redefined our relationship with our phone and in the process put disrupting the telecom industry. Flickr made a largely way web into a dynamic, thriving social community. Today even our friends at Wall Street Journal have comments, and the New York Times is hoping to use LinkedIn to foster a community. YouTube made us rethink television, by making it dead simple not only to consume video but also to broadcast video.

Similarly, Google’s Search changed how we consumed information. Instead of going to destinations, we now consume information by just finding it. What made this “big idea” even more disruptive — Google’s use of data analytics to offer highly focused advertising messages to marry search queries. (Of course, Google wasn’t the one to think of this big idea, but that’s a whole another story.)

The problem with "Big Ideas" is that sometimes people get obsessed with how game changing or disruptive their product is instead of focusing on making it the best at what it does.

I remember a few years ago when every other article out of the technology press was about how innovative every product Google created was. For a while, this created a scramble amongst various Web companies to show that they could be just as innovative as Google. Yahoo! tried to do this by buying innovation in the form of every hot startup they could get their hands on (Flickr, del.icio.us, Jumpcut, Webjay, Konfabulator, Upcoming, etc) which only led to spreading itself too thin as eloquently captured in Brad Garlinghouse's Peanut Butter Manifesto memo.

On the other hand, there are lots of products that have been unoriginal ideas now seen as game changing because of how these ideas have been executed. One example is the iPod + iTunes which started of as yet another MP3 player with a proprietary music player but has grown to be a cultural force and the number one retailer of music in the United States. When Facebook launched it plowed ground that had already been covered by MySpace, Orkut and Friendster yet today it is the number one social networking site in the world. In both cases, these have been very focused products that have been careful to grow their feature set in a limited way but have done an excellent job in providing a very user-centric experience. This isn't to say that there aren't any innovations in Facebook or in iTunes+iPod but instead that they are examples of products that have been game changing not by being a "big idea" but instead by being full of lots of good ideas.

Being original or innovative is nice but even better is building a product that your customers love. Never forget that.

Note Now Playing: Justin Timberlake - What Goes Around.../...Comes Around Interlude Note


 

A few days ago, the top news story on Techmeme was the fact that Google launched Google Friend Connect and Facebook announced the final version of Facebook Connect within minutes of each other. Reading up on both announcements it seems interesting to note how most of the coverage is about who will win the race to dominate the Web as opposed to what end user value is actually being created by these technologies.

It was somewhat depressing until I read Dave Winer's brilliant post Soon it will be time to start over, again which contains the following excerpt

We're now reaching the end of a cycle, we're seeing feature wars. That's what's going on between Facebook and Google, both perfectly timing the rollouts of their developer proposition to coincide with the others' -- on the very same day! I don't even have to look at them and I am sure that they're too complicated. Because I've been around this loop so many times. The solution to the problem these guys are supposedly working on won't come in this generation, it can only come when people start over. They are too mired in the complexities of the past to solve this one. Both companies are getting ready to shrink. It's the last gasp of this generation of technology.  Permalink to this paragraph

But the next one can't be far away now. It will be exhilirating!! Permalink to this paragraph

Remember how great Google was when it first appeared? Permalink to this paragraph

Remember how great Netscape was, and before that Apple, and I know you guys won't like this, but Microsoft offered us some great new places to play. I remember finding out that their OS address space in 1981 was 640K. That was a lot to guy who was spending huge amounts of time trying to cram a 256K app into 48K. Permalink to this paragraph

The trick in each cycle is to fight complexity, so the growth can keep going. But you can't keep it out, engineers like complexity, not just because it provides them job security, also because they really just like it. But once the stack gets too arcane, the next generation throws their hands up and says "We're not going to deal with that mess."  Permalink to this paragraph

We're almost there now. ;-> Permalink to this paragraph

The value of Facebook Connect to Facebook is obvious. They get to become a centralized identity provider for the Web including the benefit of tracking every single time one of their users logs-in on a partner which lets them build an even better advertising profile of their users. Similarly the value to the customers of the sites adopting it seem clear at first. Below are the claimed benefits of Facebook Connect to users from my initial perusal

  1. One login, multiple sites. No need to create a new account on partner sites.
  2. Account information such as profile picture, location and other fields on the partner site can be prepopulated from Facebook
  3. Bring your social graph with you to partner sites. 
  4. Let your friends on Facebook know what you are doing on partner sites. Updates show up on your profile but do not go in your friends' news feeds (they go in their live feed instead). 

Where things get interesting is that none of these benefits require a proprietary and centralized approach like Facebook has done. If Facebook implemented OpenID and OpenID attribute exchange, they could have given their users the benefits of #1 and #2 using widely adopted industry standards.  For #3, there is the burgeoning Portable Contacts effort to define a set of standard APIs for accessing the social graph that supports the key data portability principles around this information. As for broadcasting your updates from one site to another, FriendFeed has shown how that can be done using standard technologies like RSS, Atom and XMPP. 

Ignoring the fact that Facebook Connect is a proprietary and centralized approach instead of being based on open standards, there are still other points worthy of debate. When trying out sites like CitySearch beta with Facebook Connect, the experience is that I am connected with all of my Facebook friends who also use CitySearch. There is the genuine question of whether users really want to use one friends' list across every site regardless of context (e.g. interacting with the exact same people on LinkedIn, MySpace and XBox Live) or whether they want to have universal access to any of their friends lists and bridge them when necessary?

Yesterday on Twitter, I mentioned that Facebook Connect is the wrong direction to go on the Web for the reasons above. I also called Google Friend Connect a misguided "me too" effort for trying to copy Facebook's strategy and glomming an AJAX widget play on top of it. Kevin Marks, an evangelist at Google challenged by statement with the following response

@Carnage4Life the problem for users is re-entering data and restating friends for each app. For developers its becoming social without this

If that is truly the problem, how does the technology in the video below solve the problem any better than the combination of OpenID and Portable Contacts?

As with OpenSocial, Google has fallen in love with its role as a spoiler when it comes to Facebook's platform efforts without stopping to think whether it actually makes sense to be aping Facebook's strategies in the first place. Monkey see, monkey do.

This will be the death of them if they aren't careful. Once you become a follower and define yourself by reacting to others actions, it is hard to step back into a leadership role both in the industry and even within one's corporate culture.

Note Now Playing: Britney Spears - Circus Note


 

My RSS reader is buzzing with a lot hype around Facebook's Connect this morning. The lead story seems to be the New York Times article entitled Facebook Aims to Extend Its Reach Across the Web which announces that a number of popular sites are about to adopt the technology. The article is excerpted below

Facebook Connect, as the company’s new feature is called, allows its members to log onto other Web sites using their Facebook identification and see their friends’ activities on those sites. Like Beacon, the controversial advertising program that Facebook introduced and then withdrew last year after it raised a hullabaloo over privacy, Connect also gives members the opportunity to broadcast their actions on those sites to their friends on Facebook.

In the next few weeks, a number of prominent Web sites will weave this service into their pages, including those of the Discovery Channel and The San Francisco Chronicle, the social news site Digg, the genealogy network Geni and the online video hub Hulu.

MySpace, Yahoo and Google have all announced similar programs this year, using common standards that will allow other Web sites to reduce the work needed to embrace each identity system. Facebook, which is using its own data-sharing technology, is slightly ahead of its rivals.

This set of partners is definitely higher profile than the last list of Facebook Connect adopters and yet I still have to wonder how this is eventually going to shake out. Even with this set of partners there are still two big hurdles Facebook has to surmount. The first is just getting users to connect their identities on different sites with their Facebook identity. Just having the ability to connect a Digg account and a Facebook account doesn't mean users will adopt the feature. I assume this is why the Facebook Beacon automatically linked a user's account on a partner site to their Facebook account in the first place. How this is presented to users on participating sites will be key to its adoption and this is mostly out of Facebook's control.

The other challenge that Facebook Connect will face is how to prevent it from tarred with the same "centralized identity service" brush that Microsoft's Passport got tarred with at the turn of the century. Back in the year 2000, Joel Spolsky wrote Does Issuing Passports Make Microsoft a Country? which began as follows

Am I the only one who is terrified about Microsoft Passport? It seems to me like a fairly blatant attempt to build the world's largest, richest consumer database, and then make fabulous profits mining it. It's a terrifying threat to everyone's personal privacy and it will make today's "cookies" seem positively tame by comparison. The scariest thing is that Microsoft is advertising Passport as if it were a benefit to consumers, and people seem to be falling for it! By the time you've read this article, I can guarantee that I'll scare you into turning off your Hotmail account and staying away from MSN web sites.

These sentiments never went away and by 2005 Microsoft had lost some of its most prominent Passport partner sites. The service has since been rebranded Windows Live ID and is now primarily a unified identity system for Microsoft sites as opposed to being a single-sign on service for the entire Web. It may be that Microsoft was ahead of its time (as I've argued it was with Hailstorm and other initiatives) but the arguments against centralized identity systems have seemed pretty convincing in the past. In addition, I suspect that developers will start asking questions when they realize that they have to support one proprietary technology for Facebook Connect, something different for MySpace Data Availability and yet another for Google Friend Connect. How many sign-in buttons will end up adorning these sites? http://beta.citysearch.com already has two sign-in links, will that expand to four as they move to support signing in with your MySpace account and your Google Friend Connect-enabled account? Or will services decide to pick a social network to side with to the exclusion of all others? It's beginning to remind me of the high definition DVD format wars and not in a good way.

Interesting times indeed.

Note Now Playing: Kanye West - Welcome To Heartbreak (feat. Kid Cudi) Note


 

John McCrea of Plaxo has written a cleverly titled guest post on TechCrunchIT, Facebook Connect and OpenID Relationship Status: “It’s Complicated”, where he makes the argument that Facebook Connect is a competing technology to OpenID but the situation is complicated by Facebook developers engaging in discussions with the OpenID. He writes

You see, it’s been about a month since the first implementation of Facebook Connect was spotted in the wild over at CBS’s celebrity gossip site, TheInsider.com. Want to sign up for the site? Click a single button. A little Facebook window pops up to confirm that you want to connect via your Facebook account. One more click – and you’re done. You’ve got a new account, a mini profile with your Facebook photo, and access to that subset of your Facebook friends who have also connected their accounts to TheInsider. Oh, and you can have your activities on TheInsider flow into your Facebook news feed automatically. All that, without having to create and remember a new username/password pair for the site. Why, it’s just like the vision for OpenID and the Open Stack – except without a single open building block under the hood!
...
After the intros, Allen Tom of Yahoo, who organized the event, turned the first session over Max Engel of MySpace, who in turn suggested an alternative – why not let Facebook’s Julie Zhuo kick it off instead? And for the next hour, Julie took us through the details of Facebook Connect and the decisions they had to make along the way to get the user interface and user experience just right. It was not just a presentation; it was a very active and engaged discussion, with questions popping up from all over the room. Julie and the rest of the Facebook team were engaged and eager to share what they had learned.

What the heck is going on here? Is Facebook preparing to go the next step of open, switching from the FB stack to the Open Stack? Only time will tell. But one thing is clear: Facebook Connect is the best thing ever for OpenID (and the rest of the Open Stack). Why? Because Facebook has set a high bar with Facebook Connect that is inspiring everyone in the open movement to work harder and faster to bring up the quality of the UI/UX for OpenID and the Open Stack.

There are a number of points worth discussing from the above excerpt. The first is the implication that OpenID is an equivalent technology to Facebook Connect. This is clearly not the case. OpenID just allows you to delegate to act of authenticating a user to another website so the user doesn't need to create credentials (i.e. username + password) on your site. OpenID alone doesn't get you the user's profile data nor does it allow you to pull in the authenticated user's social graph from the other site or publish activities to their activity feed. For that, you would need other other "Open brand" technologies like OpenID Attribute Exchange, Portable Contacts and OpenSocial. So it is fairer to describe the contest as Facebook Connect vs. OpenID + OpenID Attribute Exchange + Portable Contacts + OpenSocial.

The question then is who should we root for? At the end of the day, I don't think it makes a ton of sense for websites to have to target umpteen different APIs that do the same thing instead of targeting one standard implemented by multiple services. Specifically, it seems ridiculous to me that TheInsider.com will have to code against Facebook Connect to integrate Facebook accounts into their site but code against something else if they want to integrate MySpace accounts and yet another API if they want to integrate LinkedIn accounts and so on. This is an area that is crying out for standardization.

Unfortunately, the key company providing thought leadership in this area is Facebook and for now they are building their solution with proprietary technologies instead of de jure or de facto ("Open brand") standards. This is unsurprising given that it takes three or four different specs in varying states of completeness created by different audiences deliver the scenarios they are currently interested in. What is encouraging is that Facebook developers are working with OpenID implementers by sharing their knowledge. However OpenID isn't the only technology needed to satisfy this scenario and I wonder if Facebook will be similarly engaged with the folks working on Portable Contacts and OpenSocial.

Facebook Connect is a step in the right direction when it comes to bringing the vision of social network interoperability to fruition. The key question is whether we will see effective open standards emerge that will target the same scenarios [which eventually even Facebook could adopt] or whether competitors will offer their own proprietary alternatives? So far it sounds like the latter is happening which means unnecessary reinvention of the wheel for sites that want to support "connecting" with multiple social networking sites.

PS: If OpenID phishing is a concern now when the user is redirected to the ID provider's site to login, it seems Facebook Connect is even worse since all it does is provide a pop over. I wonder if this is because the Facebook folks think the phishing concerns are overblown.

Note Now Playing: 2Pac - Mind Made Up (feat. Daz, Method Man & Redman) Note


 

John Battelle has a blog post entitled When Doesn't It Pay To Pay Attention To Search Quality? which contains the following statement and screenshot

the top result is the best result - and it's a paid link.

Bad Results

In the past I've talked about Google's strategy tax which is the conflict between increasing the relevance of their search results and increasing the relevance of their search ads. The more relevant Google's "organic" results are the less likely users are to click on their ads which means the less money the company makes. This effectively puts a cap on how good Google's search quality can get especially given the company's obsessive measurements of every small change they make to get the most bang for the buck.

When I first wrote about this, the conclusion from some quarters was that this inherent conflict of interest would eventually be Google's undoing since there were search innovations that they would either be slow to adopt or put on the back burner so as not to harm their cash cow. However John Battelle's post puts another spin on the issue. As long as people find what they want it doesn't matter if the result is "organic" or an ad.

As Jeremy Zawodny noted in his post The Truth about Web Navigation 

Oh, here's a bonus tip: normal people can't tell the difference between AdSense style ads and all the other links on most web sites. And almost the same number don't know what "sponsored results" on the Search Results Page are either. It's just a page of links to them. They click the ones that look like they'll get them what they want. It's that simple.

even more interesting is the comment by Marshall Kirkpatrick in response to Jeremy's post

The part of your post about AdWords reminds me of a survey I read awhile ago. Some tiny percentage of users were able to tell the difference between paid and natural search results, then once away from the computer almost all of them when asked said that the best ways to make it clear would be: putting paid links in a colored box, putting them in a different section of the page and putting the words "sponsored links" near them!! lol

What this means in practice is that the relevance of Google's ads in relation to the search term will increase in comparison to the relevance of the organic search results for that term. John Battelle has shown one example of this in his blog post. Over time this trend will get more pronounced. The problem for Google's competitors is that this doesn't necessarily mean that their search experience will get worse over time since their ad relevance will likely make up for any deficiencies in their organic results (at least for commercial queries – where the money is). What competitors will have to learn to exploit is Google's tendency to push users to Adwords results by making their organic results satisfactory instead of great.  

For example, consider the following search results page which my wife just got while looking for an acupuncturist in Bellevue, Washington

The interesting thing about the organic results is that it is relevant but very cluttered thus leading to the paradox of choice. On the other hand, the sponsored links give you a name and a description of the person's qualifications in their first result. Which result do you think my wife clicked?

Now why do you think Google ended up going with this format for commercial/business-based search results?  The average Web user is a satisficer and will look for the clearest and simplest result which is in the ads. However the geeks and power users (i.e. people who don't click on ads) are often maximizers when it comes to search results and are thus served by the organic results.

The question for you is whether you'd consider this a weakness or a strength of Google Search?

Note Now Playing: Missy Elliott - Hit Em Wit Da Hee (remix) (feat. Lil Kim & Mocha) Note


 

For a hot, pre-IPO startup I'm surprised that Facebook seems to be going through an exodus of co-founders (how many do they have?) and well-respected key employees. Just this year alone we've seen the following exits

If you combine the list above with the New York Times article, The Facebooker Who Friended Obama which states

Mr. [Chris] Hughes, 24, was one of four founders of Facebook. In early 2007, he left the company to work in Chicago on Senator Obama’s new-media campaign. Leaving behind his company at such a critical time would appear to require some cognitive dissonance: political campaigns, after all, are built on handshakes and persuasion, not computer servers, and Mr. Hughes has watched, sometimes ruefully, as Facebook has marketed new products that he helped develop.

That's three departures from people named as co-founders of Facebook. Of course, it isn't unusual for co-founders to leave a business they helped start even if the business is on the path to being super successful (see Paul Allen) but it is still an interesting trend nonetheless.

Note Now Playing: T.I. - My Life, Your Entertainment (Ft. Usher) Note


 

September 15, 2008
@ 04:12 PM

A few weeks ago Google released the beta of Google Chrome, a new Web browser based on WebKit. Since then there has been a lot of interesting hype and backlash against the hype about Chrome. Two great examples of the hype and the corresponding backlash are Mike Arrington's Meet Chrome, Google’s Windows Killer and Ted Dziuba's article Chrome-fed Googasm bares tech pundit futility in response.

The best way to think about Google Chrome is to understand how Google thinks about the Web. Nick Carr has a post entitled The Omnigoogle which does a great job of capturing a sentiment I've seen expressed by every Google employee I've ever talked to from senior people like Sergey Brin and Vint Cerf to front line folks Dewitt Clinton and Kevin Marks. Nick Carr writes

But while Google is an unusual company in many ways, when you boil down its business strategy, you find that it’s not quite as mysterious as it seems. The way Google makes money is straightforward: It brokers and publishes advertisements through digital media. More than 99 percent of its sales have come from the fees it charges advertisers for using its network to get their messages out on the Internet.

Google’s protean appearance is not a reflection of its core business. Rather, it stems from the vast number of complements to its core business. Complements are, to put it simply, any products or services that tend be consumed together. Think hot dogs and mustard, or houses and mortgages. For Google, literally everything that happens on the Internet is a complement to its main business. The more things that people and companies do online, the more ads they see and the more money Google makes. In addition, as Internet activity increases, Google collects more data on consumers’ needs and behavior and can tailor its ads more precisely, strengthening its competitive advantage and further increasing its income. As more and more products and services are delivered digitally over computer networks — entertainment, news, software programs, financial transactions — Google’s range of complements expands into ever more industry sectors. That's why cute little Google has morphed into The Omnigoogle.

Because the sales of complementary products rise in tandem, a company has a strong strategic interest in reducing the cost and expanding the availability of the complements to its core product. It’s not too much of an exaggeration to say that a company would like all complements to be given away. If hot dogs became freebies, mustard sales would skyrocket. It’s this natural drive to reduce the cost of complements that, more than anything else, explains Google’s strategy.

This boils down to the corporate ideology that "anything that is good for the Web is good for Google". This means Google is in favor of anything that increases the breadth of the Web which explains why it is investing in O3b networks in an effort intended to bring the Web to 3 billion people in emerging markets. The more people there are using the Web, the more people there are viewing ads on Google's services and on pages of sites that use AdSense and DoubleClick ads. This also means that Google is in favor of moving as much media consumption as possible to the Web. This explains why purchasing YouTube was so important. In addition to purchasing the number one video site on the Web, Google also ensured that it would be on the front line of defending video on the Web given that YouTube was in the cross hairs of various corporate content owners. This focus on expanding the breadth of the Web also explains why they have purchased startups like Zenter, Upstartle and 2Web Technologies to create a Google office suite in an attempt to unseat the current breed of desktop based office productivity software. It explains why they created Gmail as a way to make Web-based email as satisfying or even more satisfying than desktop mail experiences especially when compared to other Webmail offerings at the time. This ideology also explains why the company invests in Android and so on..

The media has tried to make it seem like Google spits out a bunch of random, unfocused projects without much thought besides "shipping something cool". However this is far from the case. Google is the most successful company on the Web and it believes that its fortunes are directly tied to the increased usage and evolution of the Web. This means Google has a strong incentive to improve the capabilities of the Web as a delivery vehicle for user experiences. Google had telegraphed their intent to take a more direct role in the evolution of Web technologies in a few ways. For one, the company hired Ian Hickson who had been rallying browser vendors to start improving Web technologies like HTML via the Web Hypertext Applications Technology Working Group (WHAT WG). His success in these efforts since joining Google has led to HTML 5 becoming an official W3C effort. Secondly, Google also heavily supported Firefox both by hiring developers who worked on Firefox full time and via a search affiliate program that brings in millions for the Mozilla corporation [Ed note – Google has a similar deal with Opera]. However the relationship with Firefox clearly was not evolving the Web at a pace that Google found satisfactory as evidenced by the creation of Google Gears a product which Google evangelists have positioned as a bleeding edge HTML 5 implementation even though it implements capabilities not mentioned in HTML 5. 

However even with having a seat at the table in defining HTML 5 and being a significant sponsor of the second most popular Web browser, Google still did not have a direct way to push the evolution of the Web directly to users. They were still dependent on the pace of innovation of incumbent browser vendors or figuring out how to distribute a browser plug-in by convincing companies like MySpace to take a dependency on it. This was clearly an uphill battle. Thus creating their own Web browser was inevitable.

So why is this significant? It isn't because "Google Chrome is going to replace Windows" or some other such silliness. As it stands now, Google Chrome is a Windows based application whose most interesting features exist in other browsers. A Web browser cannot replace an operating system any more than an automobile can replace an Interstate highway. The significant end user innovation in Google Chrome is that it is bundled with Google Gears. This means that Google Chrome has a mechanism for delivering richer experiences to end users out of the box. Google can now use this as a carrot and a stick approach to convincing browser vendors to do what it wants. Google can make its sites work better together with Chrome + Gears (e.g. YouTube Uploader using Gears) which could lead to lost browser market share for competing browser vendors if this becomes a widespread practice among Google's offerings. Even if Google never does this, the implied threat is now out there.

Chrome will likely force Google's competitors to up their game with regards to adopting newer Web standards and features just to stay competitive. This is similar to what Google did with online mapping and Web mail, and what the Opera browser has been doing by pioneering features like "pr0n mode" and tabbed browsing. So even if Google loses because Chrome doesn't get massively popular, Google still wins because the user experience for browsing the Web has been improved.  And at the end of the day, if more people are using the Web because the user experience is better across the board that's just fine for Google. The same way the fact that all online mapping experiences and Web mail experiences have improved across the board is also good for Google.

Now Playing: Metallica - The Judas Kiss


 

Update: A blog post on the official Google blog entitled A fresh take on the browser states that the comic book went out early. It was supposed to show up tomorrow in tandem with the launch of the beta of Google Chrome which will be available in over 100 countries.

Phillipp Lessen was sent an announcement in the form of a comic book which gives the details on an upcoming Open Source browser project from Google. He gives the details and links to scanned images of the comic in his post Google Chrome, Google’s Browser Project. His site seems to have been running slow so he took down links to the scans. I managed to grab them and have uploaded them to a folder on Windows Live SkyDrive. The link is embedded below and the comic can be accessed directly from here.

The key features of Chrome according to the comic are

  • Based on WebKit
  • Each browser tab gets it's own process and Javascript execution is actually multithreaded instead of single threaded as it is in most modern browsers. This actually solves lots of memory problems with modern browsers since it reduces memory fragmentation and mitigates the impact of memory leaks since all memory is reclaimed when the bad Website's browser tab is closed.
  • Task manager shows how much resources each tab is using so you can tie resource usage to individual Web pages.
  • Created Javascript VM from scratch which has clever optimizations like just-in-time compilation and incremental garbage collection.
  • Each tab has it's own URL box and can effectively be considered it's own browser Window.
  • OmniBox URL bar is similar to AwesomeBar in Firefox 3 and the smart address bar in IE 8.
  • Instead of "about:blank" the default homepage shows our nine most visited Web pages and four most used search engines
  • Has an incognito mode where no browser history is saved and cookies are wiped out when the browser is closed. Some people have affectionately dubbed such features pr0n mode. Amusingly, the comic uses the same "planning a surprise birthday party" scenario that the Internet Explorer team used when describing a similar feature in IE 8. 
  • Pop ups are not modal and scoped to the tab which they were spawned from. However they can be promoted to becoming their own tab.
  • There is a "streamlined" mode where the URL box and browser toolbar are hidden so only the Web page is visible.
  • Web pages are sandboxed so that if the user hits a malware site it cannot access the user's computer and perform drive-by downloads or installations.
  • The sandbox model is broken by browser plugins (e.g. Flash) but this is mitigated by having the plugin execute in it's own separate process that is different from that of the browser's rendering engine.
  • The browser will continuously phone home to Google to get a list of known malware sites so that the user is warned when they are visited.
  • Will ship with Google Gears built-in

My initial thoughts on this are that this is a smart move on Google's part. Google depends on the usage and advancement of the Web for its success. However how quickly the Web is advanced is in the hands of browser vendors which probably doesn't sit well with them which is why they created Gears in the first place and hired the guy driving HTML 5. Chrome now gives them an even larger say in which way the Web goes.

As for their relationship with Firefox, it may be mutually beneficial financially since Mozilla gets paid while Google gets to be the search default in a popular browser but it doesn't mean Google can dictate their technical direction. With Chrome, has a way to force browser vendors to move the Web forward on their terms even if it is just by getting users to demand the features that exist in Chrome.

PS: Am I the only one that thinks that Google is beginning to fight too many wars on too many fronts. Android (Apple), OpenSocial (Facebook), Knol (Wikipedia), Lively (IMVU/SecondLife), Chrome (IE/Firefox) and that's just in the past year.

Now Playing: Boys Like Girls - Thunder


 

According to Werner Vogels's blog post entitled Amazon EBS - Elastic Block Store has launched, it seems that my friends at Amazon have plugged a gaping hole in their cloud computing platform story. Werner writes

Back in the days when we made the architectural decision to virtualize the internal Amazon infrastructure one of the first steps we took was a deep analysis of the way that storage was used by the internal Amazon services. We had to make sure that the infrastructure storage solutions we were going to develop would be highly effective for developers by addressing the most common patterns first. That analysis led us to three top patterns:

  1. Key-Value storage. The majority of the Amazon storage patterns were based on primary key access leading to single value or object. This pattern led to the development of Amazon S3.
  2. Simple Structured Data storage. A second large category of storage patterns were satisfied by access to simple query interface into structured datasets. Fast indexing allows high-speed lookups over large dataset. This pattern led to the development of Amazon SimpleDB. A common pattern we see is that secondary keys to objects stored in Amazon S3 are stored in SimpleDB, where lookups result in sets of S3 (primary) keys.
  3. Block storage. The remaining bucket holds a variety of storage patterns ranging special file systems such as ZFS to applications managing their own block storage (e.g. cache servers) to relational databases. This category is served by Amazon EBS which provides the fundamental building block for implementing a variety of storage patterns.

What I like about Werner's post is that it shows that Amazon had a clear vision and strategy around providing hosted cloud services and has been steadily executing on it.

S3 handled what I've typically heard described as "blob storage". A typical Web application typically has media files and other resources (images, CSS stylesheets, scripts, video files, etc) that is simply accessed by name/path. However a lot of these resources also have metadata (e.g. a video file on YouTube has metadata about it's rating, who uploaded it, number of views, etc) which need to be stored as well. This need for queryable, schematized storage is where SimpleDB comes in. EC2 provides a virtual server that can be used for computation complete with a local file system instance which isn't persistent if the virtual server goes down for any reason. With SimpleDB and S3 you have the building blocks to build a large class of "Web 2.0" style applications when you throw in the computational capabilities provided by EC2.

However neither S3 nor SimpleDB provides a solution for a developer who simply wants the typical LAMP or WISC developer experience of building a database driven Web application or for applications that may have custom storage needs that don't fit neatly into the buckets of blob storage or schematized storage. Without access to a persistent filesystem, developers on Amazon's cloud computing platform have had to come up with sophisticated solutions involving backing data up manually from EC2 to S3 to get the desired experience.

EBS is the final piece in the puzzle that had prevented Amazon's cloud computing platform from being comparable to traditional hosting solutions. With EBS Amazon is now superior to most traditional hosting solutions from a developer usability perspective as well as cost. Google App Engine now looks like a plaything in comparison. In fact, you could build GAE on top of Amazon's cloud computing platform now that the EBS has solved persistent custom storage problem. It will be interesting to see if higher level cloud computing platforms such as App Engine start getting built on top of Amazon's cloud computing platform. Simply porting GAE wholesale would be an interesting academic exercise and a fun hacking project. 

Now Playing: T.I. - Whatever You Like


 

It is a fairly well known fact in the business community that the majority of mergers and acquisitions are a failure when it comes to increasing shareholder value, benefiting customers or any of the other metrics that are used to judge the "success" of an acquisition. Whenever you read a news story about some startup being acquired or two large companies merging, there is a greater than 50% chance that the resulting product or company will be of less value to customers and shareholders than if the deal had never happened.

When it comes to software company acquisitions, there are additional factors working against success that go beyond the typical laundry list of reasons that are given for why M&As usually result in failure. With technology company acquisitions not only are there a minefield of people and financial issues that have to be dealt with, there is also the real problem of what to do about technology mismatch that often exists across different companies.

Whenever a large software company acquires a startup, the first order of business is often an attempt to move the startup's application onto the larger company's technology infrastructure so that it can get benefits of "economies of scale" or some other buzzword that is typically a euphemism for "we bought you so now you're our bitches" that is not grounded in business realities. This often requires application rewrites that have the unfortunate consequence of causing the shipped application to stagnate as all efforts are poured into recreating the same application using a different technology. In addition, the founders of the startup typically get frustrated with what they [rightfully] deem as a pointless exercise and eventually move on to greener pastures. There are a number of examples of this that have occurred in the "Web 2.0" space as shown below

From Fred Wilson's post We Need A New Path To Liquidity

So if you can't take a company public, how do you get out? M&A has been the primary answer in the web/tech sector for the past eight years. And it's been a great period to sell companies. We've sold three in the past couple years out of our Union Square Ventures portfolio, delicious, FeedBurner, and TACODA, to Yahoo!, Google, and AOL, respectively. Were we happy to take their money? Yes. Were we happy with the outcome? Yes. Were they good buys for their new owners? On the face of it, yes.

But if you look deeper, I wonder. Delicious grew nicely for a while under Yahoo!'s ownership but recently the user base has fallen off pretty dramatically. I double checked this chart in compete and alexa and they all show the dropoff.

Delicious

Well, what about FeedBurner? Clearly Google has done a good job with that acquisition. Well I am not sure. I don't see any integration between Adwords and FeedBurner yet. I can't buy FeedBurner inventory through Google's text ad interface. I honestly don't see any additional money flowing to me, the publisher of the feed, since the Google acquisition. There's no way to know what the rate of signup by publishers has been since the acquisition, but I wonder if it's increased much.

And TACODA? I know that TACODA had an incredible fourth quarter post the acquisition by AOL, blowing way past the numbers we were projecting in our annual budget. But in the first quarter, AOL fired Curt Viebranz, TACODA's CEO, and many of the top members of the TACODA team are now gone from AOL. Another acquisition messed up.

But who am I to complain? We got paid right? So sit down and shut up.

From Joshua Schachter's comment on "How Yahoo dropped the del.icio.us ball with a pointless 3 year rewrite (from mod_perl to PHP)"

The writer is accidentally correct - we were told that it had to be in PHP to get ops support. Curiously the PHP part didn't take that much time - the majority of the "business logic" is in C++ which took forever and ever to write. I think the open question now is whether the remaining team will be able to innovate or be stuck in complicated codebase hell. 

From Dennis Howlett's article Google Sites - spoiled by usability issues

After 16 months at Google developer’s hands, the outcome is substandard. This is such a pity. In its JotSpot incarnation, it was far from perfect but that didn’t matter because JotSpot was shedding light on a new way of collaborating. Since passing into Google’s hands, the guts have been ripped out and then re-assembled with as much Google ’stuff’ as they could cram in but rushed to completion.

At the very least, Google should get rid of the gadgets addition facility and rework it. Otherwise, I sense the SMBs at which it is aimed will find the service a turn off.

Google has a real chance to differentiate itself from Microsoft - which is clearly what it wants to do, while adding significant numbers of users to its Google Apps offering. It won’t do it this way because despite all the gripes around Microsoft products, the fact is Microsoft offers a more polished experience. Until Google truly understands this, it will find it difficult to adequately compete. In the meantime, offerings like Wetpaint and Ning have little to fear.

From Ryan Paul's article Jaiku users flee to Twitter as a result of Google's neglect

Unfortunately, Google has allowed Jaiku to languish and is now suffering a backlash from frustrated users who are beginning to mass-migrate to Twitter, a competing microblogging service. Jaiku's external feed servers, which are used by third-party Jaiku client applications, have been down frequently during the past week, often returning 504 gateway errors or nothing at all. During the brief stints when the feed servers are operational, they have been extremely slow and often out of sync with the actual content—typically lagging by between four and 13 hours. These problems have been noted by many users and several third-party Jaiku client application developers who discussed the problem with Ars. Users also complain that Jaiku's IM bots and the third-party Jaiku Facebook interface are exhibiting problems as well.

When Google announced the acquisition, the company promised new features within a few months, but we have seen no evidence of any development at all. Registration is still closed and new users can only join the site by receiving an invitation from Google. The Jaiku developers have been completely and totally silent since the announcement of the takeover, and the official Jaiku blog—which used to have several messages a month—has had no new posts at all. The Jaiku Team feed has also not received any posts from Jaiku developers since the acquisition.

The stories are the same except that some of the names are different. A startup gets bought and immediately stops innovating because all their development time is being spent porting the code to a new platform. During that time newer, more agile competitors show up and eat their lunch. Why I find this to be such a conundrum is that when you buy a technology startup, you are primarily buying three things

  • customers
  • employees
  • technology

However the standard operating procedure during Web software acquisitions is to discard the technology and consequentially tick off the employees who made the product a success in the first place thus creating an exodus. The application rewrite plus employee exodus leads to product stagnation which eventually leads to lots of pissed off customers. Thus the entire value from the acquisition is effectively thrown away.

This is the default situation when it comes to acquisitions in the software industry. For every successful acquisition like Google + YouTube there are two or three that are more like Google + Dodgeball. So if there is a startup whose product you love that you hear is being acquired by a one of the large Web companies, be happy for the founders and be sad for yourself because the product you love is likely going to become a neglected bride.

Disclaimer: I'm an employee of a large software company that has displayed similarly counterproductive tactics when acquiring startups. Although no examples are provided in the post above, I'm sure some can be found from it's history of acquisitions.

Now Playing: Dire Straits - Money For Nothing


 

Earlier today I read two posts that were practically mirror opposites of each other. The first was Paul Graham's essay, The Pooled-Risk Company Management Company, where he makes a case against founding a company that you intend to run as a successful business over the long term and instead building a company you that you can sell to a public company so you can move on and enjoy your money. His argument is excerpted below

At this year's startup school, David Heinemeier Hansson gave a talk in which he suggested that startup founders should do things the old fashioned way. Instead of hoping to get rich by building a valuable company and then selling stock in a "liquidity event," founders should start companies that make money and live off the revenues. Sounds like a good plan. Let's think about the optimal way to do this.

One disadvantage of living off the revenues of your company is that you have to keep running it. And as anyone who runs their own business can tell you, that requires your complete attention. You can't just start a business and check out once things are going well, or they stop going well surprisingly fast.

The main economic motives of startup founders seem to be freedom and security. They want enough money that (a) they don't have to worry about running out of money and (b) they can spend their time how they want. Running your own business offers neither. You certainly don't have freedom: no boss is so demanding. Nor do you have security, because if you stop paying attention to the company, its revenues go away, and with them your income.

The best case, for most people, would be if you could hire someone to manage the company for you once you'd grown it to a certain size.
...
If such pooled-risk company management companies existed, signing up with one would seem the ideal plan for most people following the route David advocated. Good news: they do exist. What I've just described is an acquisition by a public company.

Austin Wiltshire has a counterpoint to Paul Graham's article entitled New hire cannon fodder where he decries the practice of "exploiting" junior developers so that a couple of fat cats with money can get even richer. Parts of Austin's counter argument are excerpted below

Why these large firms, and now even places like YCombinator continually think the best way to move forward in software is to hire as many gullible young naive programmers as possible and work them to death is beyond me.  It’s pretty well known that 80 hour work weeks and inexperience is a guarantee to continually make the same damn mistakes over and over again.  It’s also an open question as to why new hires let these companies take advantage of them so badly.  Paul Graham had a start up, he begged for angel investing, and his life should show you - what does he do now?  Well he learned from his experience that designing and building is for chumps, to make the big bucks and sit on your ass you become an angel investor.

Kids will work for pennies.  You can continue to fill their heads with dreams of having the next big idea, even though they are carrying all the risk for you.  Junior developers, whether entrepreneurs or otherwise, are being asked to give up their 20’s, probably the best, most energetic years of their lives, to have a chance at making a dent in someone else’s bottom line.  (Make note, the one exception here I’ve seen is 37 Signals :) )

But have we never stopped to think who truly is benefiting from all these hours?  Do we get paid more?  No.  In fact, because many of us are salaried, we’re effectively paid less.  Are we compensated with faster promotions?  Possibly - but don’t forget about that silicon ceiling.  The only person who knows how many hours you’re putting in is probably just the guy above you - but he makes sure to show just how productive his department is (via your hard work) to everyone.  He will always get the spoils.  Who will end up really getting the spoils out of any of YCombinator’s work?  Paul Graham.

Both arguments have their merits and there are also parts I disagree with on both sides. Austin is right that YCombinator takes advantage of the naivety of youth. However when you are in your 20s with no serious attachments (like a mortgage, a family or even a sick relative) it doesn't sound like a bad idea to make hay while the sun shines. If you can sacrifice some time in your youth for a chance at a better life for yourself and your future spouse/kids/girlfriend/family/etc in a few years, is it wrong to treat that as an opportunity? Especially if you'll be working in an energetic environment surrounded by likeminded souls all believing that you are building cool stuff? Additionally, if the startup doesn't work out [which it most likely won't] the experience will still turn out to be useful when you decide to get a regular job at some BigCo even if it is just realizing how good you have it to no longer have to work 80 hour weeks while eating Top Ramen for breakfast and dinner any more.

From that perspective I don't think Austin is right to completely rail against the startup lifestyle. However I totally agree with the general theme of Austin's post that working ridiculous hours is dumb. It should be common knowledge that sleep deprivation impairs brain function and may even lead to psychiatric disorders. The code you check-in during your 14th hour at your desk will not be as good as what you checked in during your 4th. If you really have to work that much, work on the weekends instead of spending over 12 hours sitting in front of your IDE. Even then, busting your butt to that extent only makes sense if you not only get to share the risks of failure but also the rewards of success as well. This means you better be a co-founder or someone with equity and not just some poor sap on salary.

My issue with Paul Graham's essay and the investment style of YCombinator is that I it sells startup founders short. Paul recently wrote an essay entitled Cities and Ambition where he had this beautiful quote about the kind of "peer pressure" the Silicon Valley area exerts on startup founders

When you ask what message a city sends, you sometimes get surprising answers. As much as they respect brains in Silicon Valley, the message the Valley sends is: you should be more powerful.

That's not quite the same message New York sends. Power matters in New York too of course, but New York is pretty impressed by a billion dollars even if you merely inherited it. In Silicon Valley no one would care except a few real estate agents. What matters in Silicon Valley is how much effect you have on the world. The reason people there care about Larry and Sergey is not their wealth but the fact that they control Google, which affects practically everyone.

Read the above quote again and let its message sink in. The great thing about software is how you can literally take nothing (i.e. a blank computer screen) and build something that changes the world. Bill and Paul did it with Microsoft. Larry and Sergey have done it with Google. Jerry and David did it with Yahoo!, and some might say Mark Zuckerberg is doing it with Facebook.

Are any of those companies YCombinator-style, built-to-flip companies? Nope.

I strongly believe in the idea behind the mantra "Change the World or Go Home". Unlike anything that has come before it, the combination of software and the World Wide Web has the potential to connect people and empower them in more ways than humanity has never seen. And it is possible to become immensely rich while moving humanity forward with the software that you create.

So if you have decided to found a startup, why decide to spend your youth building some "me too" application that conforms to all the current "Web 2.0" fads in the desperate hope that you can convince some BigCo to buy you out? That sounds like such a waste. Change the world or go home.

Now Playing: Wu-Tang Clan - Triumph


 

July 28, 2008
@ 02:27 AM

For several months Nick Carr has pointed out that Wikipedia ranks highly in the search results for a number of common topics in Google's search engine. In his post entitled Googlepedia Nick Carr speculated on why Google would see this trend as a threat in a paragraph which is excerpted below

I'm guessing that serving as the front door for a vast ad-less info-moshpit outfitted with open source search tools is not exactly the future that Google has in mind for itself. Enter Knol.

Clearly Nick Carr wasn't the only one that realized that Google was slowly turning into a Wikipedia redirector. Google wants to be the #1 source for information or at least be serving ads on the #1 sites on the Internet in specific area. Wikipedia was slowly eroding the company's effectivenes at achieving both goals. So it is unsurprising that Google has launched Knol and is trying to entice authors away from Wikipedia by offering them a chance to get paid.

What is surprising is that Google is tipping it's search results to favor Knol. Or at least that is the conclusion of several search engine optimization (SEO) experts and also jibes with my experiences.

Danny Sullivan wrote in his post The Day After: Looking At How Well Knol Pages Rank On Google that

We've been assured that just because content sits on Google's Knol site, it won't gain any ranking authority from being part of the Knol domain. OK, so a day after Knol has launched, how's that holding up? I found 1/3 of the pages listed on the Knol home page that I tested ranked in the top results.

I was surprised to see a post covering how Knol's How to Backpack was already hitting the number three spot on Google. Really? I mean, how many links could this page have gotten already? As it turns out, quite a few. And more important, it's featured on the Knol home page, which itself is probably one of the most important links. While Knol uses nofollow on individual knols to prevent link credit from flowing out, it's not used on the home page -- so home page credit can flow to individual knols featured on it.

here's a test knol I made yesterday -- Firefox Plugins For SEO & SEM -- which ranks 28 for firefox plugins for seo. I never linked to it from my article about knol. I don't think it made the Knol home page. I can see only three links pointing at it, and only one of those links uses anchor text relevant to what the page is ranking for. And it's in the top 30 results?

Look, I know that being ranked 28 is pretty much near invisible in terms of traffic you'll get from search engines. But then again, to go from nowhere to the 28th top page in Google out of 755,000 matches? I'm sorry -- don't tell me that being in Knol doesn't give your page some authority.

Aaron Wall noticed something even more insidious in his post entitled Google Knol - Google's Latest Attack on Copyright where he notices that if Google notices duplicate content then it favors the content on Knol over a site that has existed for years and has decent PageRank. His post is excerpted below

Another Knol Test

Maybe we are being a bit biased and/or are rushing to judgement? Maybe a more scientific effort would compare how Knol content ranks to other content when it is essentially duplicate content? I did not want to mention that I was testing that when I created my SEO Basics Knol, but the content was essentially a duplicate of my Work.com Guide to Learning SEO (that was also syndicated to Business.com). Even Google shows this directly on the Knol page

Google Knows its Duplicate Content

Is Google the Most Authoritative Publisher?

Given that Google knows that Business.com is a many year old high authority directory and that the Business.com page with my content on it is a PageRank 5, which does Google prefer to rank? Searching for a string of text on the page I found that the Knol page ranks in the search results.

If I override some of Google's duplicate content filters (by adding &filter=0 to the search string) then I see that 2 copies of the Knol page outrank the Business.com page that was filtered out earlier.

Interesting.

Following Danny's example, I also tried running some searches for terms that appear on the Knol homepage and seeing how they did in Google's search. Here's the screenshot of the results of searching for "buttermilk pancakes"

Not bad for a page that has existed on the Web for less than two weeks.

Google is clearly favoring Knol content over content from older, more highly linked sites on the Web. I won't bother with the question of whether Google is doing this on purpose or whether this is some innocent mistake. The important question is "What are they going to do about it now that we've found out?"

Now Playing: One Republic - Stop and Stare
 

I've been using the redesigned Facebook profile and homepage for the past few days and thought it would be useful to write up my impressions on the changes. Facebook is now the the world's most popular social networking site and one of the ways they've gotten there is by being very focused on listening to their users and improving their user experienced based on this feedback. Below are screenshots of the old and new versions of the pages and a discussion of which elements are changed and the user scenarios the changes are meant to improve.

Homepage Redesign

OLD HOME PAGE:

NEW HOME PAGE:

The key changes and their likely justifications are as follows

  • Entry points for creating content are now at the top of the news feed. One of the key features driving user engagement on Facebook is the News Feed. This lets a user know what is going on with their social network as soon as they logon to the site. In a typical example of network effects at work, one person creates some content by uploading a photo or sharing a link and hundreds of people on their friend list benefit by having content to view in their News Feed. If any of the friends responds to the content this again benefits hundreds of people and so on.  The problem with the old home page was that a user sees their friends uploading photos and sharing links and may want to do so as well but there is no easy way for her to figure out how to do the same thing without having to go two or three clicks away from the home page. The entry points at the top of the feed will encourage more "impulse" content creation.

  • Left sidebar is gone. There were three groups of items in the left nav; a search box, the list of a user's most frequently accessed applications and an advertisement. The key problem is that the ad is in a bottom corner of the feed. This makes it easy for users to mentally segregate that part of the screen from their vision and either never look there or completely ignore it. Removing that visual ghetto and moving ads to being inline with the feed makes it more likely that users will look at the ad. Ah, but now you need more room to show the ad (all the space isn't needed for news feed stories). So the other elements of the left nave are moved, the search box to the header and the list of most accessed applications to the sidebar on the right. Now you have enough room to stretch out the News Feed's visible area and advertisers can reuse their horizontal banner ads on Facebook even though this makes the existing feed content now look awkward. This is one place where monetization trumped usability.

  • Comments now shown inline for News Feed items with comments (not visible in screen shot). This may be the feature that made Mike Arrington decide to call the new redesign the FriendFeedization of Facebook. Sites like FriendFeed have proven that showing the comments on an item in the feed inline gives users more content to view in their feeds and increases the likelihood of engagement since the user may want to join the conversation.

Profile Redesign

OLD PROFILE:

NEW PROFILE:

The key changes and their likely justifications are as follows

  • The profile now has tabbed model for navigation. This is a massive improvement for a number of reasons. The most important one is that in the old profile, there is a lot of content below the fold. My old profile page is EIGHT pages when printed as opposed to TWO pages when the new profile page is printed. Moving to a tabbed model (i) improves page load times and (ii) increases number of page views and hence ad impressions.

  • The Mini-Feed and the Wall have been merged. The intent here is to give more visibility to the Wall which in the old model was below the fold. The "guest book" or wall is an important part of the interaction model in social networking sites (see danah boyd's Friendster lost steam. Is MySpace just a fad? essay) and Facebook was de-emphasizing theirs in the old model.

  • Entry points for creating content are at the top of the profile page. Done for the same reason as on the Home page. You want to give users lots of entry points for adding content to the site so that they can kick off network effects by generating content which in turn generates tasty page views.

  • Left sidebar is gone. Again the left sidebar is gone and the advertisement is moved closer to the content, and away from the visual ghetto that is the bottom left of the screen. Search box and most accessed applications are now in the header as well. The intent here is also to improve the likelihood that users will view and react to the ads.

Now Playing: Da Back Wudz - I Don't Like The Look Of It (Oompa)


 

For the past few years, the technology press has been eulogizing desktop and server-based software while proclaiming that the era of Software as a Service (SaaS) is now upon us. According to the lessons of the Innovator's Dilemma the cheaper and more flexible SaaS solutions will eventually replace traditional installed software and the current crop of software vendors will turn out to be dinosaurs in a world that belongs to the warm blooded mammals who have conquered cloud based services.

So it seems the answer is obvious, software vendors should rush to provide Web-based services and extricate themselves from their "legacy" shrinkwrapped software business before it is too late. What could possibly go wrong with this plan? 

Sarah Lacy wrote an informative article for Business Week about the problems facing software vendors who have rushed into the world of SaaS. The Business Week article is entitled On-Demand Computing: A Brutal Slog and contains the following excerpt

On-demand represented a welcome break from the traditional way of doing things in the 1990s, when swaggering, elephant hunter-style salesmen would drive up in their gleaming BMWs to close massive orders in the waning days of the quarter. It was a time when representatives of Oracle (ORCL), Siebel, Sybase (SY), PeopleSoft, BEA Systems, or SAP (SAP) would extol the latest enterprise software revolution, be it for management of inventory, supply chain, customer relationships, or some other area of business. Then there were the billions of dollars spent on consultants to make it all work together—you couldn't just rip everything out and start over if it didn't. There was too much invested already, and chances are the alternatives weren't much better.

Funny thing about the Web, though. It's just as good at displacing revenue as it is in generating sources of it. Just ask the music industry or, ahem, print media. Think Robin Hood, taking riches from the elite and distributing them to everyone else, including the customers who get to keep more of their money and the upstarts that can more easily build competing alternatives.

But are these upstarts viable? On-demand software has turned out to be a brutal slog. Software sold "as a service" over the Web doesn't sell itself, even when it's cheaper and actually works. Each sale closed by these new Web-based software companies has a much smaller price tag. And vendors are continually tweaking their software, fixing bugs, and pushing out incremental improvements. Great news for the user, but the software makers miss out on the once-lucrative massive upgrade every few years and seemingly endless maintenance fees for supporting old versions of the software.

Nowhere was this more clear than on Oracle's most recent earnings call (BusinessWeek.com, 6/26/08). Why isn't Oracle a bigger player in on-demand software? It doesn't want to be, Ellison told the analysts and investors. "We've been in this business 10 years, and we've only now turned a profit," he said. "The last thing we want to do is have a very large business that's not profitable and drags our margins down." No, Ellison would rather enjoy the bounty of an acquisition spree that handed Oracle a bevy of software companies, hordes of customers, and associated maintenance fees that trickle straight to the bottom line.

SAP isn't having much more luck with Business by Design, its foray into the on-demand world, I'm told. SAP said for years it would never get into the on-demand game. Then when it sensed a potential threat from NetSuite, SAP decided to embrace on-demand. Results have been less than stellar so far. "SAP thought customers would go to a Web site, configure it themselves, and found the first hundred or so implementations required a lot of time and a lot of tremendous costs," Richardson says. "Small businesses are calling for support, calling SAP because they don't have IT departments. SAP is spending a lot of resources to configure and troubleshoot the problem."

In some ways, SaaS vendors have been misled by the consumer Web and have failed to realize that they still need to spend money on sales and support when servicing business customers. Just because Google doesn't advertise it's search features and Yahoo! Mail doesn't seem to have a huge support staff that hand holds customers as it uses their product doesn't mean that SaaS vendors can expect to cut their sales and support calls. The dynamics of running a free, advertising based service aimed at end users is completely different from running a service where you expect to charge business tens of thousands to hundreds of thousands to use your product.

In traditional business software development you have three major cycles with their own attendant costs; you have to write the software, you have to market the software and then you have to support the software. Becoming a SaaS vendor does not eliminate any of these costs. Instead it adds new costs and complexities such as managing data centers and worrying about hackers. In addition, thanks to free advertising based consumer services and the fact that companies like Google that have subsidized their SaaS offerings using their monopoly profits in other areas, business customers expect Web-based software to be cheaper than its desktop or server-based alternatives. Talk about being stuck between a rock and a hard place as a vendor.

Finally, software vendors that have existing ecosystems of partners that benefit from supporting and enhancing their shrinkwrapped products also have to worry about where these partners fit in a SaaS world. For an example of the kinds of problems these vendors now face, below is an excerpt from a rant by Vladimer Mazek, a system administrator at ExchangeDefender, entitled Houston… we have a problem which he wrote after attending one of Microsoft's partner conferences

Lack of Partner Direction: By far the biggest disappointment of the show. All of Microsoft’s executives failed to clearly communicate the partnership benefits. That is why partners pack the keynotes, to find a way to partner up with Microsoft. If you want to gloat about how fabulous you are and talk about exciting commission schedules as a brand recommender and a sales agent you might want to go work for Mary Kay. This is the biggest quagmire for Microsoft – it’s competitors are more agile because they do not have to work with partners to go to market. Infrastructure solutions are easy enough to offer and both Google and Apple and Amazon are beating Microsoft to the market, with far simpler and less convoluted solutions. How can Microsoft compete with its partners in a solution ecosystem that doesn’t require partners to begin with?

This is another example of the kind of problems established software vendors will have to solve as they try to ride the Software as a Service wave instead of being flattened by it.  Truly successful SaaS vendors will eventually have to deliver platforms that can sustain a healthy partner ecosystems to succeed in the long term. We have seen this in the consumer space with the Facebook platform and in the enterprise space with SalesForce.com's AppExchange. Here is one area where the upstarts that don't have a preexisting shrinkwrap software businesses can turn a disadvantage (lack of an established partner ecosystem) into an advantage since it is easier to start from scratch than to retool.

The bottom line is that creating a Web-based version of a popular desktop or server-based product is just part of the battle if you plan to play in the enterprise space. You will have to deal with the sales and support that go with selling to businesses as well as all the other headaches of shipping "cloud based services" which don't exist in the shrinkwrap software world. After you get that figured out, you will want to consider how you can leverage various ISVs and startups to enhance the stickiness of your service and turn it into a platform before one of your competitor's does. 

I suspect we still have a few years before any of the above happens. In the meantime we will see lots more software companies complaining about the paradox of embracing the Web when it clearly cannibalizes their other revenue streams and is less lucrative than what they've been accustomed to seeing. Interesting times indeed.

Now Playing: Flobots - Handlebars


 

About a week ago, the Facebook Data team quietly released the Cassandra Project on Google Code. The Cassandra project has been described as a cross between Google's BigTable and Amazon's Dynamo storage systems. An overview of the project is available in the SIGMOD presentation on Cassandra available at SlideShare. A summary of the salient aspects of the project follows.

The problem Cassandra is aimed at solving is one that plagues social networking sites or any other service that has lots of relationships between users and their data. In such services, data often needs to be denormalized to prevent having to do lots of joins when performing queries. However this means the system needs to deal with the increased write traffic due to denormalization. At this point if you're using a relational database, you realize you're pretty much breaking every major rule of relational database design. Google tackled this problem by coming up with BigTable. Facebook has followed their lead by developing Cassandra which they admit is inspired by BigTable. 

The Cassandra data model is fairly straightforward. The entire system is a giant table with lots of rows. Each row is identified by a unique key. Each row has a column family, which can be thought of as the schema for the row. A column family can contain thousands of columns which are a tuple of {name, value, timestamp} and/or super columns which are a tuple of {name, column+} where column+ means one or more columns. This is very similar to the data model behind Google's BigTable.

As I mentioned earlier, denormalized data means you have to be able to handle a lot more writes than you would if storing data in a normalized relational database. Cassandra has several optimizations to make writes cheaper. When a write operation occurs, it doesn't immediately cause a write to the disk. Instead the record is updated in memory and the write operation is added to the commit log. Periodically the list of pending writes is processed and write operations are flushed to disk. As part of the flushing process the set of pending writes is analyzed and redundant writes eliminated. Additionally, the writes are sorted so that the disk is written to sequentially thus significantly improving seek time on the hard drive and reducing the impact of random writes to the system. How important is improving seek time when accessing data on a hard drive? It can make the difference between taking hours versus days to flush a hundred gigabytes of writes to a disk. Disk is the new tape.

Cassandra is described as "always writable" which means that a write operation always returns success even if it fails internally to the system. This is similar to the model exposed by Amazon's Dynamo which has an eventual consistency model.  From what I've read, it isn't clear how writes operations that occur during an internal failure are reconciled and exposed to users of the system. I'm sure someone with more knowledge can chime in in the comments.

At first glance, this is a very nice addition to the world of Open Source software by the Facebook team. Kudos.

Found via James Hamilton.

PS: Is it me or is this the second significant instance of Facebook Open Sourcing a key infrastructure component "inspired" by Google internals?

Now Playing: Ray J - Gifts


 

Matt Asay of C|Net has an article entitled Facebook adopts the CPAL poison pill where he writes

Instead, by choosing CPAL, Facebook has ensured that it can be open source without anyone actually using its source. Was that the intent?

As OStatic explains, CPAL requires display of an attribution notice on derivative works. This practice, which effectively requires downstream code to carry the original developer(s)' logo, came to be known as "badgeware." It was approved by the OSI but continues to be viewed with suspicion within the open-source community.

I've written before about how most open-source licenses don't apply themselves well to the networked economy. Only the OSL, AGPL, and CPAL contemplate web-based services. It's not surprising that Facebook opted for one of these licenses, but I am surprised it chose the one least likely to lead to developers actually modifying the Facebook platform.

If the point was to protect the Facebook platform from competition (i.e., derivative works), Facebook chose a good license. If it was to encourage development, it chose the wrong license.

But if the purpose was to prevent modifications of the platform, why bother open sourcing it at all?

I've seen more than one person repeat the sentiment in the above article which leaves me completely perplexed. With fbOpen Facebook has allowed anyone who is interested to run Facebook applications and participate in what is currently the most popular & vibrant social network widget ecosystem in the world.

I can think of lots of good reasons for not wanting to adopt fbOpen. Maybe the code is in PHP and you are a Ruby On Rails shop. Or maybe it conflicts with your company's grand strategy of painting Facebook as the devil and you the heroes of openness (*cough* Google *cough*). However I can't see how requiring that you mention somewhere on your site that your social network's widget platform is powered by the Facebook developer platform is some sort of onerous POISON PILL which prevents you from using it. In the old days, companies used to charge you for the right to say your application is compatible with theirs, heck, Apple still does. So it seems pretty wacky for someone to call Facebook out for letting people use their code and encouraging them to use the Facebook brand in describing their product. Shoot!

The premise of the entire article is pretty ridiculous, it's like calling the BSD License a poison pill license because of the advertising clause. This isn't to say there aren't real issues with an advertising clause as pointed out in the GNU foundation's article The BSD License Problem. However as far as I'm aware,  adopters of fbOpen don't have to worry about being obligated to display dozens powered by X messages because every bit of code they depend on requires that it is similarly advertised. So that argument is moot in this case.

Crazy article but I've come to expect that from Matt Asay's writing.

Now Playing: Eminem & D12 - Sh*t On You


 

Recently the folks behind Twitter came clean on the architecture behind the service and it is quite clear that the entire service is being held together by chewing gum and baling wire. Only three MySQL database servers for a service that has the I/O requirements of Twitter? Consider how that compares to other Web 2.0 sites that have come clean with their database numbers; Facebook has 1800, Flickr has 166, even Wikipedia has 20. Talk about bringing a knife to a gunfight.

Given the fact that Twitter has had scaling issues for over a year it is surprising that not only has it taken so long for them to figure out that they need a re-architecture but more importantly they decided that having a developer/sys admin manage fail over and traffic spikes by hand was cheaper to the business than buying more hardware and a few weeks of coding. 

A popular social networking that focuses on features instead of performance while upstart competitors are waiting in the wings? Sounds like a familiar song doesn't it? This entire episode reminds me of a story I read in the New York Times a few years ago titled The Wallflower at the Web Party which contains the following familiar sounding excerpts

But the board also lost sight of the task at hand, according to Kent Lindstrom, an early investor in Friendster and one of its first employees. As Friendster became more popular, its overwhelmed Web site became slower. Things would become so bad that a Friendster Web page took as long as 40 seconds to download. Yet, from where Mr. Lindstrom sat, technical difficulties proved too pedestrian for a board of this pedigree. The performance problems would come up, but the board devoted most of its time to talking about potential competitors and new features, such as the possibility of adding Internet phone services, or so-called voice over Internet protocol, or VoIP, to the site.
...
In retrospect, Mr. Lindstrom said, the company needed to devote all of its resources to fixing its technological problems. But such are the appetites of companies fixated on growing into multibillion-dollar behemoths. They seek to run even before they can walk.

“Friendster was so focused on becoming the next Google,” Professor Piskorski said, “that they weren’t focused on fixing the more mundane problems standing in the way of them becoming the next Google.”
...
“We completely failed to execute,” Mr. Doerr said. “Everything boiled down to our inability to improve performance.”

People said about Friendster the same thing they say about Twitter, we travel in tribes - people won't switch to Pownce or Jaiku because all their friends use Twitter. Well Friendster thought the same thing until MySpace showed up and now we have Facebook doing the same to them.

It is a very vulnerable time for Twitter and a savvy competitor could take advantage of that by adding a few features while courting the right set of influential users to jumpstart an exodus. The folks at FriendFeed could be that competitor but I suspect they won't. The Bret & Paul have already boxed their service into being an early adopter's play thing when there's actually interesting mainstream potential for their service. They'd totally put paid to their dreams of being a household brand if they end up simply being a Twitter knock off even if they could end up outplaying Evan and Biz at the game they invented.

Now Playing: Bob Marley - Redemption Song


 

Disclaimer: This post does not reflect the opinions, thoughts, strategies or future intentions of my employer. These are solely my personal opinions. If you are seeking official position statements from Microsoft, please go here.

Recently there were three vaporware announcements by Facebook, Google and MySpace each describing a way for other web sites to integrate the user profiles and friends lists from these popular social networking sites. Given that I'm a big fan of social networking sites and interoperability between them, this seemed like an interesting set of announcements. So I decided to take a look at these announcements especially given the timing of them.  

What Do They Have in Common?

Marc Canter does a good job of describing the underlying theme behind all three announcements in his post I do not compromise where he writes

three announcements that happened within a week of each other: MySpace’s Data Availability, Facebook’s Connect and Google’s Friend Connect - ALL THREE had fundamentally the same strategy!

They’re all keeping their member’s data on their servers, while sending out tentacles to mesh in with as many outside sites as they can. These tentacles may be widgets, apps or iFrames - but its all the same strategy.

Basically all three announcements argue that instead of trying to build social networking into their services from scratch, Web sites should instead outsource their social graphs and "social features" such as user profiles, friends lists and media sharing from the large social networking sites like Facebook, MySpace and Orkut.

This isn't a new pitch, Facebook has been singing the same song since they announced the beta of the Facebook Platform in August 2006 and Google has been sending Kevin Marks to every conference they can find to give his Social Cloud presentation which makes the same pitch. The new wrinkle to this time worn tale is that Google and Facebook [along with MySpace] are no longer just pitching using REST APIs for integration but are now preaching "no coding required" integration via widgets. 

Now that we know the meat of all three announcements we can go over the little specifics that have leaked out about each forthcoming product thus far.

Facebook Connect

Dave Morin gave the first official statement about Facebook Connect news in his blog post Announcing Facebook Connect where he wrote

Trusted Authentication
Users will be able to connect their Facebook account with any partner website using a trusted authentication method. Whether at login, or anywhere else a developer would like to add social context, the user will be able to authenticate and connect their account in a trusted environment. The user will have total control of the permissions granted.

Real Identity
Facebook users represent themselves with their real names and real identities. With Facebook Connect, users can bring their real identity information with them wherever they go on the Web, including: basic profile information, profile picture, name, friends, photos, events, groups, and more.

Friends Access
Users count on Facebook to stay connected to their friends and family. With Facebook Connect, users can take their friends with them wherever they go on the Web. Developers will be able to add rich social context to their websites. Developers will even be able to dynamically show which of their Facebook friends already have accounts on their sites.

Dynamic Privacy
As a user moves around the open Web, their privacy settings will follow, ensuring that users' information and privacy rules are always up-to-date. For example, if a user changes their profile picture, or removes a friend connection, this will be automatically updated in the external website.

The key features to note are (i) a user can associate their Facebook account with their account on a 3rd party site which means  (ii) the user's profile and media shared on Facebook can now be exposed on the 3rd party site and (iii) the users friends' on Facebook who have also associated their Facebook account with their account on the 3rd party site will show up as the user's friends on the site. 

The "dynamic privacy" claim seems pretty vague if not downright empty. All that is stated above is that the user's changes on Facebook are instantly reflected on 3rd party sites. Duh. Does that need to be called out as a feature?

Google Friend Connect

On the Google Friend Connect page there is the following video

The key features to note are (i) a user can associate their Facebook account OpenID with their account on a 3rd party site which means  (ii) the user's profile and media shared on Facebook account a small set of social networking site can now be exposed on the 3rd party site and (iii) the users friends' on Facebook the small set of social network sites who have also associated their Facebook account OpenID using Google Friend Connect to connect their account on the 3rd party site will show up as the user's friends on the site (iv) the user's activities on the 3rd party site are broadcast in her friends' news feeds.

One interesting thing about Google Friend Connect's use of OpenID is that it allows me to associate multiple social network profiles to a single account which may not even be from a social networking site (e.g. using my AOL or Y! email to sign-in but associating it with my Facebook profile & friend list).

Google Friend Connect seems to be powered by Google OpenSocial which is Google's attempt to commoditize the functionality of the Facebook platform by making it easy for any social networking site to roll its own Facebook-style platform by using Google's standard set of REST APIs, Javascript libraries and/or hosting services. In the above video, it is mentioned that Web sites which adopt Google Friend Connect will not only be able to obtain user profile and friend list widgets from Google but also OpenSocial widgets written by 3rd party developers. However since Facebook announced the JavaScript Client Library for Facebook API way back in January they already have the technology in place to offer something similar to Web site owners if this capability becomes in demand.  More important will be the set of functionality that comes "out of the box" so to speak since a developer community won't form until Google Friend Connect gains traction.

By the way, it turns out that Facebook has banned Google from interacting with their user data using Google Friend Connect since it violates their terms of service. My assumption is that the problem is Google Friend Connect works by building an OpenSocial wrapper on top of the Facebook API and then exposing it to other web sites as widgets and to OpenSocial gadget developers via APIs. Thus Google is pretty much proxying the Facebook social graph to other sites and developers which takes control of safeguarding/policing access to this user data out of Facebook's hands. Not good for Facebook. 

MySpace Data Availability

The only details on the Web about MySpace's Data Availability seems to be second hand data from tech bloggers who were either strategically leaked some details/screenshots or took part in a press release conference call. The best source I found was Mike Arrington's TechCrunch post entitled MySpace Embraces DataPortability, Partners With Yahoo, Ebay And Twitter which contains the following excerpt

image

MySpace is announcing a broad ranging embrace of data portability standards today, along with data sharing partnerships with Yahoo, Ebay, Twitter and their own Photobucket subsidiary. The new project is being called MySpace “Data Availability” and is an example, MySpace says, of their dedication to playing nice with the rest of the Internet.

A mockup of how the data sharing will look in action with Twitter is shown above. MySpace is essentially making key user data, including (1) Publicly available basic profile information, (2) MySpace photos, (3) MySpaceTV videos, and (4) friend networks, available to partners via their (previousy internal) RESTful API, along with user authentication via OAuth.

The key goal is to allow users to maintain key personal data at sites like MySpace and not have it be locked up in an island. Previously users could turn much of this data into widgets and add them to third party sites. But that doesn’t bridge the gap between independent, autonomous websites, MySpace says. Every site remains an island.

But with Data Availability, partners will be able to access MySpace user data, combine it with their own, and present it on their sites outside of the normal widget framework. Friends lists can be syncronized, for example. Or Twitter may use the data to recommend other Twitter users who are your MySpace friends.

The key difference between MySpace's announcement and those of Facebook & Google is that MySpace has more ground to cover. Since Facebook & Google already have REST APIs that support a delegated authentication model, MySpace is pretty much playing catch up here.

In fact, on careful rereading it seems MySpace's announcement isn't like the others since the only concrete technology announced above is a REST API that uses a user-centric delegated authentication model which is something both Google and Facebook have had for years (see GData/OpenSocial and Facebook REST API).

Given my assumption that MySpace is not announcing anything new to the industry, the rest of this post will focus on Google Friend Connect and Facebook Connect.  

The Chicken & Egg Problem

When it comes to social networking, it is all about network effects. A social networking feature or site is only interesting to me if my friends are using it as well.

The argument that a site is better off using a user's social graph from a big social networking site like Facebook instead of building their own social network features only makes sense if (i) there is enough overlap in the user's friends list on Facebook and that on the site AND (ii) the user's friends on the site who are also his friends on Facebook can be discovered by the user. The latter is the tough part and one I haven't seen a good way of bridging without resorting to anti-patterns (i.e. pull the email addresses of all of the user's friends from Facebook and then cross-reference with the email addresses of the sites users). This anti-pattern works when you are getting the email addresses the user entered by hand from some Webmail address book (e.g. Hotmail, Gmail, Y! mail, etc).

However since Google and Facebook are going with a no-code solution, the only way to tell which of my Facebook friends also use the 3rd site is if they have also opted-in to linking their account on the site with their Facebook profile. This significantly weakens the network effects of the feature compared to the find your friends on your favorite "Web 2.0" site which a lot of sites have used to grow their user base by screen scraping Webmail address books then cross referencing it with their user databases.

How Does this Relate to Data Portability and Social Network Interoperability?

Short answer; it doesn't.

Long answer; the first thing to do is to make sure you understand what is meant by Data Portability and Social Network Interoperability. The difference between Data Portability and Social Network Interoperability is the difference between being able to export your email inbox and address book from Gmail into Outlook or vice versa (portable) and being able to send an email from a Gmail address to someone using Outlook or Hotmail (interoperable).

So do these new widget initiatives help portability? Nope. Widgets give developers less options for obtaining and interacting with the user data than APIs. With Facebook's REST API, I know how to get my friends list with profile data into Outlook and my Windows Mobile phone via OutSync. I would actually lose that functionality if it was only exposed via a widget. The one thing they do is lower the bar for integration by people who don't know how to code.

Well, how about interoperability? The idea of social network interoperability is that instead of being a bunch of walled gardens and data silos, social networking sites can talk to each other the same way email services and [some] IM services can talk to each other today. The "Use our data silo instead of building your own" pitch may reduce the number of data silos but it doesn't change the fact that the Facebooks and MySpaces of the world are still fundamentally data silos when it comes to the social graph. That is what we have to change. Instead we keep getting distracted along the way by shiny widgets.

PS: The blog hiatus is over. It was fun while it lasted. ;)

Now Playing: Fugees (Refugee Camp) - Killing Me Softly


 

News of the layoffs at Yahoo! has now hit the presses. A couple of the folks who've been indicated as laid off are people I know are great employees either via professional interaction or by reputation. The list of people who fit this bill so far are Susan Mernitt, Bradley HorowitzSalim Ismail and Randy Farmer.  Salim used to run Yahoo's "incubation" wing so this is a pretty big loss. It is particularly interesting that he volunteered to leave the company which may be a coincidence or could imply that some of the other news about Yahoo! has motivated some employees to seek employment elsewhere. It will be interesting to see how this plays out in the coming months.

Randy Farmer is also a surprise given that he pretty much confirmed that he was working on Jerry Yang's secret plan for a Yahoo comeback which included

  • Rethinking the Yahoo homepage
  • Consolidating Yahoo's plethora of social networks
  • Opening up Yahoo to third parties with a consistent platform similar to Facebook's
  • Revamping Yahoo's network infrastructure

If Yahoo! is reducing headcount by letting go of folks working on various next generation projects, this can't bode well for the future of the company given that success on the Web depends on constant innovation.

PS: To any ex-Yahoo's out there, if the kind of problems described in this post sound interesting to you, we're always hiring. Give me a holler. :) 


 

From the press release entitled Microsoft Proposes Acquisition of Yahoo! for $31 per Share we learn

REDMOND, Wash. — Feb. 1, 2008 — Microsoft Corp. (NASDAQ:MSFT) today announced that it has made a proposal to the Yahoo! Inc. (NASDAQ:YHOO) Board of Directors to acquire all the outstanding shares of Yahoo! common stock for per share consideration of $31 representing a total equity value of approximately $44.6 billion. Microsoft’s proposal would allow the Yahoo! shareholders to elect to receive cash or a fixed number of shares of Microsoft common stock, with the total consideration payable to Yahoo! shareholders consisting of one-half cash and one-half Microsoft common stock. The offer represents a 62 percent premium above the closing price of Yahoo! common stock on Jan. 31, 2008.

“We have great respect for Yahoo!, and together we can offer an increasingly exciting set of solutions for consumers, publishers and advertisers while becoming better positioned to compete in the online services market,” said Steve Ballmer, chief executive officer of Microsoft. “We believe our combination will deliver superior value to our respective shareholders and better choice and innovation to our customers and industry partners.”

“Our lives, our businesses, and even our society have been progressively transformed by the Web, and Yahoo! has played a pioneering role by building compelling, high-scale services and infrastructure,” said Ray Ozzie, chief software architect at Microsoft. “The combination of these two great teams would enable us to jointly deliver a broad range of new experiences to our customers that neither of us would have achieved on our own.”

WOW. Just...wow.

There's a conference call with Ray Ozzie, Steve Ballmer, Chris Liddell and Kevin Johnson in about half an hour to discuss this. This is the first time I've considered listening in on one of those.


 

Two seemingly unrelated posts flew by my aggregator this morning. The first was Robert Scoble’s post The shy Mark Zuckerberg, founder of Facebook where he talks about meeting the Facebook founder. During their conversation, Zuckerburg admits they made mistakes with their implementation of Facebook Beacon and will be coming out with an improved version soon.

The second post is from the Facebook developer blog and it is the announcement of the JavaScript Client Library for Facebook API which states

This JavaScript client library allows you to make Facebook API calls from any web site and makes it easy to create Ajax Facebook applications. Since the library does not require any server-side code on your server, you can now create a Facebook application that can be hosted on any web site that serves static HTML.

Although the pundits have been going ape shit over this on Techmeme this is an unsurprising announcement given the precedent of Facebook Beacon. With that announcement they provided a mechanism for a limited set of partners to integrate with their feed.publishTemplatizedAction API using a Javascript client library. Exposing the rest of their API using similar techniques was just a matter of time.

What was surprising to me when reading the developer documentation for the Facebook Javascript client library is the following notice to developers

Before calling any other Facebook API method, you must first create an FB.ApiClient object and invoke its requireLogin() method. Once requireLogin() completes, it invokes the callback function. At that time, you can start calling other API methods. There are two way to call them: normal mode and batched mode.

So unlike the original implementation of Beacon, the Facebook developers aren’t automatically associating your Facebook account with the 3rd party site then letting them party on your data. Instead, it looks like the user will be prompted to login before the Website can start interacting with their data on Facebook or giving Facebook data about the user.

This is a much better approach than Beacon and remedies the primary complaint from my Facebook Beacon is Unfixable post from last month.

Of course, I haven’t tested it yet to validate whether this works as advertised. If you get around to testing it before I do, let me know if it works the way the documentation implies in the comments.


 

Ari Steinberg who works on the Facebook developer platform has a blog post entitled New Rules for News Feed which states

As part of the user experience improvements we announced yesterday, we're changing the rules for how Feed stories can be published with the feed.publishTemplatizedAction API method. The new policy moving forward will be that this function should only be used to publish actions actively taken by the "actor" mentioned in the story. As an example, feed stories that say things like "John has received a present" are no longer acceptable. The product motivation behind this change is that Feed is a place to publish highly relevant stories about user activity, rather than passive stories promoting an application.

To foster this intended behavior, we are changing the way the function works: the "actor_id" parameter will be ignored. Instead the session_key used to generate the feed story will be used as the actor.


In order to ensure a high quality experience for users, starting 9am Pacific time Tuesday 22 January we may contact you, or in severe cases initiate an enforcement action, if your stories are not complying with the new policy, especially if the volume of non-complying stories is high.

If you are not a developer using the Facebook platform, it may be unclear what exactly this announcement means to end users or applications that utilize Facebook’s APIs.

To understand the impact of the Facebook announcement, it would be useful to first talk about the malicious behavior that Facebook is trying to curb. Today, an application can call feed.publishTemplatizedAction and publish a story to the user’s Mini-feed (list of all the user’s actions) which will also show up in the News Feed of the users friends. Unfortunately some Facebook applications have been publishing stories that don’t really correspond to a user taking an action. For example, when a user installs the Flixster application, Flixster not only publishes a story to all of the user’s friends saying the user has installed the application but also publishes a story to the friends of each of the user’s friends that also have Flixster installed. This means my friends get updates such as

being sent to my friends when I wasn’t actually doing anything with the Flixster application. I don’t know about you but this seems like a rather insiduous way for an application to spread “virally”.

Facebook’s attempt to curb such application spam is to require that an application have a session key that identifies the logged in user when publishing the story which implies that the user is actually using the application from within Facebook when the story is published. The problem with this remedy is that it totally breaks applications that publish stories to the Facebook News Feed when the user isn’t on the site. For example, since I have the Twitter application installed on Facebook, my Facebook friends get an update sent to their News Feeds whenever I post something new on Twitter.

The problem for Facebook is that by limiting a valid usage of the API, they may have closed off a spam vector but have also closed off a valuable integration point for third party developers and for their users.

PS: There might be an infinite session key loophole to the above restriction which I’m sure Facebook will close off if apps start abusing it as well.

Now playing: Silkk the Shocker - It Ain't My Fault


 

I’ve read a number of stories this week that highlight that interoperability between social networking sites will be a “top ask” in 2008 (as we say at Microsoft). Earlier this week I read the Wired article Should Web Giants Let Startups Use the Information They Have About You? which does a good job of telling both sides of the story when it comes to startups screen scraping importing user data such as social graphs (i.e. friend and contact lists) from more successful sites as a way to bootstrap their social networks. The Wired article is a good read if you want to hear all sides of the story when it comes to the issue of sharing user social data between sites.

Yesterday, I saw Social Network Aggregation, Killer App in 2008? which points out the problem that users often belong to multiple social networks at once and that bridging between them is key. However I disagree with the premise that this points to need for a “Social Network Aggregator” category of applications. I personally believe that the list of 20 or so Social Network Aggregators on Mashable are all companies that would cease to exist if the industry got off it’s behind and worked towards actual interoperability between social networking sites.

Today, I saw saw Facebook disabled Robert Scoble’s account. After reading Robert’s account of the incident, I completely agree with Facebook.

Why Robert Scoble is Wrong and Facebook is Right

Here’s what Robert Scoble wrote about the incident

My account has been “disabled” for breaking Facebook’s Terms of Use. I was running a script that got them to keep me from accessing my account

I am working with a company to move my social graph to other places and that isn’t allowable under Facebook’s terms of service. Here’s the email I received:

+++++

Hello,

Our systems indicate that you’ve been highly active on Facebook lately and viewing pages at a quick enough rate that we suspect you may be running an automated script. This kind of Activity would be a violation of our Terms of Use and potentially of federal and state laws.

As a result, your account has been disabled. Please reply to this email with a description of your recent activity on Facebook. In addition, please confirm with us that in the future you will not scrape or otherwise attempt to obtain in any manner information from our website except as permitted by our Terms of Use, and that you will immediately delete and not use in any manner any such information you may have previously obtained.

The first thing to note is that Facebook allows you to extract your social graph data from their site using the Facebook platform. In fact, right now whenever I get an email from someone on my Facebook friend list in Outlook or I get a phone call from them, I see the picture from their Facebook profile. I did this using OutSync which is an application that utilizes the Facebook platform to merge data from my contacts in Outlook/Exchange with my Facebook contacts.

So if Facebook allows you to extract information about your Facebook friends via their APIs, why would Robert Scoble need to run a screen scraping script? The fact is that the information returned by the Facebook API about a user contains no contact information (no email address, no IM screen names, no telephone numbers, no street address). Thus if you are trying to “grow virally” by spamming the Facebook friend list of one of your new users about the benefits of your brand new Web 2.0 site then you have to screen scrape Facebook.  However there is the additional wrinkle that unlike address books in Web email applications Robert Scoble did not enter any of this contact information about his friends. With this in mind, it is hard for Robert Scoble to argue that the data is “his” to extract from Facebook. In addition, as a Facebook user I consider it a feature that Facebook makes it hard for my personal data to be harvested in this way. Secondly, since Robert’s script was screen scraping it means that it had to hit the site five thousand times (once for each of his contacts) to fetch all of Robert’s friends personally idenitifiable information (PII).  Given that eBay won a court injunction against Bidder’s Edge for running 100,000 queries a day, it isn’t hard to imagine that the kind of screen scraping script that Robert is using would be considered malicious even by a court of law.

I should note that Facebook is being a bit hypocritical here since they do screen scrape other sites to get the email addresses of the contacts of new users. This is why I’ve called them the Social Graph Roach Motel in the recent past. 

O’Reilly Social Graph FOO Camp

This past weekend I got an email from Tim O'Reilly, David Recordon, and Scott Kveton inviting me to a Friends of O’Reilly Camp (aka FOO Camp) dedicated to “social graph” problems. I’m still trying to figure out if I can make it based on my schedule and whether I’m really the best person to be representing Microsoft at such an event given that I’m a technical person and “social graph problems” for the most part are not technical issues.

Regardless of whether I am able to attend or not, there were some topics I wanted to recommend should be added to a list of “red herring” topics that shouldn’t be discussed until the important issues have been hashed out.

  • Google OpenSocial: This was an example of unfortunate branding. Google should really have called this “Google OpenWidgets” or “Google Gadgets for your Domain” since the goal was competing with Facebook’s widget platform not actually opening up social networks. Since widget platforms aren’t a “social graph problem” it doesn’t seem fruitful the spend time discussing this when there are bigger fish to fry.

  • Social Network Portability: When startups talk about “social network portability” it’s usually a euphemism for collecting a person’s username and password for another site, retrieving their contact/friend list and spamming those people about their hot new Web 2.0 startup. As a user of the Web, making it easier to receive spam from startups isn’t something I think should be done let alone a “problem” that needs solving. I understand that lots of people will disagree with this [even at Microsoft] but I’m convinced that this is not the real problem facing the majority of users of social networking sites on the the Web today.  

What I Want When It Comes to Social Network Interoperability

Having I’ve said what I don’t think is important to discuss when it comes to “social graph problems”, it would be rude not to provide an example fof what I think would be fruitful discussion. I wrote the problem I think we should be solving as an industry a while back in a post entitled A Proposal for Social Network Interoperability via OpenID which is excerpted below

I have a Facebook profile while my fiancée wife has a MySpace profile. Since I’m now an active user of Facebook, I’d like her to be able to be part of my activities on the site such as being able to view my photos, read my wall posts and leave wall posts of her own. I could ask her to create a Facebook account, but I already asked her to create a profile on Windows Live Spaces so we could be friends on that service and quite frankly I don’t think she’ll find it reasonable if I keep asking her to jump from social network to social network because I happen to try out a lot of these services as part of my day job. So how can this problem be solved in the general case? 

This is a genuine user problem which the established players have little incentive to fix. The data portability folks want to make it easy for you to jump from service to service. I want to make it easy for users of one service to talk to people on another service. Can you imagine if email interoperability was achieved by making it easy for Gmail users to export their contacts to Yahoo! mail instead of it being that Gmail users can send email to Yahoo! Mail users and vice versa?

Think about that.

Now playing: DJ Drama - The Art Of Storytellin' Part 4 (Feat. Outkast And Marsha Ambrosius)


 

In the past three months, I’ve seen three moves by Google that highlight that not only is their strategic vision becoming more questionable but their engineering talent has also begun to show signs of being seriously deficient

This year is going to be a watershed year for the company. They are eroding a lot of the geek cred they’ve built up over the past decade. That will be hard to regain once it is lost.

In the meantime, I’ve noticed an uptick in the quiet smart folks you don’t see heralded in blogs turning down offers from Google and picking another horse when the job offers come knocking. Watershed year, indeed.

Now playing: John Legend - I Can Change (feat. Snoop Doggy Dogg)


 

Yesterday I read about the Opening up Facebook Platform Architecture. My initial thoughts are that Facebook has done what Google claimed to have done but didn't with Open Social. Facebook seems to have provided detailed specs on how to build an interoperable widget platform unlike Google who unleashed a bunch of half baked REST API specs with no details about the "widget" aspect of the platform unless you are building an Orkut application.

As I've thought about this over the past few weeks, building a widget platform that is competitive with Facebook's is hard work. Remember all those stories about OpenSocial apps being hacked in 45 minutes or less? The problem was that sites like Plaxo Pulse and Ning simply didn't think through all the ramifications of building a widget platform and bumped up against the kind of "security 101" issues that widget platforms like Netvibes, iGoogle and Live.com gadgets solved years ago.  I started to wonder exactly how many of these social networking sites will be able to keep up with the capabilities and features of platforms like Facebook's and Orkut's when such development is outside their core competency.

In fact let's take a quote from the TechCrunch story First OpenSocial Application Hacked Within 45 Minutes 

theharmonyguy says he’s successfully hacked Facebook applications too, including the Superpoke app, but that it is more difficult:

Facebook apps are not quite this easy. The main issue I’ve found with Facebook apps is being able to access people’s app-related history; for instance, until recently, I could access the SuperPoke action feed for any user. (I could also SuperPoke any user; not sure if they’ve fixed that one. Finally, I can access all the SuperPoke actions - they haven’t fixed that one, but it’s more just for fun.) There are other apps where, last I checked, that was still an issue ( e.g. viewing anyone’s Graffiti posts).

But the way Facebook setup their platform, it’s tons harder to actually imitate a user and change profile info like this. I’m sure this kind of issue could be easily solved by some verification code on RockYou’s part, but it’s not inherent in the platform - unlike Facebook. I could do a lot more like this on FB if Facebook hadn’t set things up the way they did.

At that point I ask myself, how useful is it to have the specs for the platform if you aren't l337 enough to implement it yourself? [Update: It looks like Google is well aware of this problem and has launched an Apache project called Shindig which is meant to be an Open Source widget platform that implements the Open Social APIs. This obviously indicates that Google realizes the specs are worthless and instead shipping a reusable widget platform is the way to go. It’s interesting to note that with this move Google is attempting to be a software vendor, advertising partner and competitor to the Web’s social networking sites. That must lead to some confusing internal meetings. Smile ]

For now, Facebook has definitely outplayed Google here. The most interesting part of the Facebook announcement to me is

Now we also want to share the benefits of our work by enabling other social sites to use our platform architecture as a model. In fact, we’ll even license the Facebook Platform methods and tags to other platforms. Of course, Facebook Platform will continue to evolve, but by enabling other social sites to use what we’ve learned, everyone wins -- users get a better experience around the web, developers get access to new audiences, and social sites get more applications.

it looks like Facebook plans to assert their Intellectual Property rights on anyone who clones their platform. This is one of the reasons I've found Open Social to be worrisome abuse of the term "open". Like Facebook, Google shipped specs for a proprietary platform whose copyrights, patents, etc belong to them. Any company that implements Open Social or even GData which it is built upon is using Google's intellectual property.

What's to stop Google from asserting these intellectual property rights the way Facebook is doing today? What exactly is "open" about it that makes it any less proprietary than what Facebook just announced?


 

Om Malik has a blog post entitled Zuckerberg’s Mea Culpa, Not Enough where he writes

Frankly, I am myself getting sick and tired of repeating myself about the all-important “information transmission from partner sites” aspect of Beacon. That question remains unanswered in Zuckerberg’s blog post, which upon second read is rather scant on actual privacy information. Here is what he writes:

If you select that you don’t want to share some Beacon actions or if you turn off Beacon, then Facebook won’t store those actions even when partners send them to Facebook.”

So essentially he’s saying the information transmitted won’t be stored but will perhaps be interpreted. Will this happen in real time? If that is the case, then the advertising “optimization” that results from “transmissions” is going to continue. Right!

If they were making massive changes, one would have seen options like “Don’t allow any web sites to send stories to Facebook” or “Don’t track my actions outside of Facebook” in this image below.

This is the part of Facebook's Beacon service that I consider to be unfixable which probably needs to be stated more explicitly given comments like those by Sam Ruby in his post Little Details.

The fundamental design of Facebook Beacon is that a Web site publishes information about my transactions to Facebook without my permission and then Facebook tells me what happened after the fact. This is fundamentally Broken As Designed (B.A.D.).

I read Mark Zuckerburg's Thoughts on Beacon last week and looked at the new privacy controls. Nowhere is the fundamental problem addressed.

Nothing Mark Zuckerburg wrote changes the fact that when I rent a movie from Blockbuster Online, information about the transaction is published to Facebook regardless of whether I am a Facebook user or not.  The only change Zuckerburg has announced is that I can opt out of getting nagged to have the information spammed to my friends via the News Feed. One could argue that this isn't Facebook's problem. After all, when SixApart implemented support for Facebook Beacon they didn't decide that they'd blindly publish all activities from users of TypePad to Facebook. Instead they have an opt-in model on their site which preserves their users' privacy by not revealing information to Mark Zuckerburg's company without their permission. On the flip side the Blockbuster decided to publish information about all of their customers' video rental transaction history  to Mark Zuckerburg and company, without their explicit permission, even though this violates federal law. As a Blockbuster customer, the only way around this is to stop using Blockbuster's service.

So who is to blame here? Facebook for designing a system that assumes that 3rd parties publishing private user data to them without the user's consent is OK as the default or Facebook affiliates who care so little of their customer's privacy that they give it away to Facebook in return for "viral" references to their services (aka spam)?

Now playing: Akon - Ghetto (Green Lantern remix) (feat. Notorious B.I.G. & 2Pac)


 

I often tell people at work that turning an application into a platform is a balancing act, not only do you have to please the developers on your platform BUT you also have to please the users of your application as well.

I recently joined the This has got to stop group on Facebook. If you don't use Facebook, the front page of the group is shown in the screenshot below.

POINTLESS FACEBOOK APPLICATIONS ARE RUINING FACEBOOK (167,186 Members)

I've seen a bunch of tech folks blog about being overwhelmed by Facebook app spam like Tim Bray in his post Facebook Rules and Doc Searls in Too much face(book) time. However I assumed that the average college or high school student who used the site didn't feel that way. Looks like I was wrong.

The folks at Facebook could fix this problem easily but it would eliminate a lot of the "viralness" that has been hyped about the platform. Personally, I think applications on the site have gotten to the point where the costs have begun to outweigh the benefits. The only way to tip the balance back is to rein them in otherwise it won't be long until the clean and minimal vs. cluttered and messy aesthetics stop working in their favor in comparisons with MySpace. When that happens there will be an opportunity for someone else to do the same thing to them.

On an unrelated note,  the MoveOn.org sponsored group about Facebook Beacon has 74,000 members which is less than half of the size of the This has got to stop group.  This is despite the fact that MoveOn.org has had national media attention focused on that topic. I guess it goes to show that just because a story gets a lot of hype in blogs and the press doesn't mean that it is the most important problem facing the people it actually affects.

Now playing: Jay-Z - Ignorant Shit


 

Danny Sullivan has an article in Advertising Age entitled Forget Facebook. Search Ads Are the Revolution he writes

Facebook unleashed SocialAds this month, calling it the beginning of a revolutionary, hundred-year era in advertising that will see the end of untargeted messages in mass media. If the revolution is upon us, allow me to submit the lowly search ad as the true revolutionary. For unlike social ads and most other types of advertising, search is something people want rather than something that gets in the way.  

The trusted referral is indeed a holy grail, and Facebook will offer a new way to build word-of-mouth. But how did that friend find the sweetener in the first place? What comes first -- word-of-mouth or the egg? At some point, a new product has to hatch, and those old-school brand-building channels probably will always play a crucial role. Search offers a key way for new products to emerge and be spread around. People turn to search for solutions -- ways to enjoy coffee without the calories or local coffeehouses to try. If you're not visible in search, perhaps you won't generate word-of-mouth as easily, if at all.

Search isn't revolutionary for aiding word-of-mouth, however. It's revolutionary for not "getting into" or in the way of anything. People turn to search when they have particular desires and need particular solutions.

I agree with Danny that the search advertising like AdWords is revolutionary while word-of-mouth advertising platforms like Facebook’s SocialAds are evolutionary.  With search ads, for the first time in the history of advertising people can find advertising when they are looking for it and otherwise it stays out of their way. When I search for digital camera or zune 80 it is quite likely that I’m making a purchasing decision so showing me ads related to buying these devices makes sense. On the other hand, when I search for foreach C# or XmlTextReader.NodeType I don’t get irrelevant ads shoved in my face. That level of match making between given consumers and advertisers is unprecedented. 

However this doesn’t mean that there isn’t something to be said for brand advertising and word of mouth. For search advertising to work, I actually have to have been looking for something in the first place. A lot of advertising today is intended to create the desire for a product in the first place not help you make an informed choice.  For example, I saw the movie Enchanted last weekend. I found out about the movie from TV ads and thought what I saw looked funny. My wife also came to the same conclusion from watching similar ads and then we decided to see the movie. After seeing the movie, I thought it was great and rated the movie in the Flixster Facebook application which sent out the following notification to my “friends”

Enchanted: **** out of *****

a few days later, one of my co-workers said she saw the movie on the strength of my recommendation and other factors.

This story is a small case study in the effectiveness of traditional “media-based” advertising coupled with the power of word-of-mouth marketing using social networking sites. For now, search ads simply cannot provide a similar level of return value for such advertisers. Although search engines like Google have tried to encourage this behavior, people don’t typically perform searches like movies 98052 then decide what movies to see that weekend based on the search results page. This means that for certain classes of products, traditional advertising techniques in combination with word-of-mouth techniques like Facebook’s social ads are extremely valuable.

However at the end of the day, it is extremely unlikely that improved word-of-mouth techniques will be as impactful to the long tail of advertisers as search ads have been or ever will be. 

Now playing: Tear Da Club Up Thugs - Triple Six Clubhouse


 

December 1, 2007
@ 05:24 PM

Earlier this week I wrote a blog post which pointed out that the two major privacy and user experience problems with Facebook Beacon where that it (i) linked a user's Facebook account with an account on another site without the users permission and (ii) there was no way for a user to completely opt out of being tracked by the system.  Since then Facebook has announced some changes which TechCrunch named Facebook Beacon 2.0. The changes are excerpted below

Notification

Facebook users will see a notification in the lower right corner of the screen after transacting with a Beacon Affiliate. Options include “No Thanks” that will immediately stop the transaction from being published. Alternatively closing or ignoring the warning won’t immediately publish the story, but it will be put in a queue
beacon2b.jpg

Second Warning

Presuming you’ve ignored or closed the first notification, Facebook warns users again the next time they visit their home page. A new box reminds you that an activity has been sent to Facebook. Like the first notification you can choose to not publish the activity by hitting remove, or you can choose to publish it by hitting ok.

...

Opt Out
Found via the “External Websites” section of the Facebook Privacy page, this allows users to permanently opt in or out of Beacon notifications, or if you’re not sure be notified. The downside is that there is no global option to opt out of every Beacon affiliated program; it has to be set per program. Better this than nothing I suppose.

The interesting thing to note is that neither of the significant problems with Beacon have been fixed. After the changes were announced there was a post on the CA Security Advisory blog titled Facebook's Misrepresentation of Beacon's Threat to Privacy: Tracking users who opt out or are not logged in which pointed out that the complaining about purchase history getting into the news feed of your friends is a red herring, the real problem is that once a site signs up as a Facebook affiliate they begin to share every significant action you take on the site with Facebook without your permission. 

Which is worse, your friends knowing that you rented Prison Girls or Facebook finding that out without your permission and sharing that with their business partners, without your permission? Aren't there laws against this kind of invasion of privacy? I guess there are (see 18 U.S.C. § 2710)

I wonder who'll be first to sue Facebook and Blockbuster? 

Anyway, back to the title of this blog post. The problem with Facebook Beacon is that it is designed in a way that makes it easy for Facebook Beacon affiliates to integrate into their sites at the cost of user's privacy. From Jay Goldman's excellent post where he Deconstructed the Facebook Beacon Javascript we learn

Beacon from 10,000 Feet

That basically wraps up our tour of how Beacon does what it does. It's a fairly long explanation, so here's a quick summary:

  1. The partner site page includes the beacon.js file, sets a <meta> tag with a name, and then calls Facebook.publish_action.            
  2. Facebook.publish_action builds a query_params object and then passes it to Facebook._send_request.            
  3. Facebook._send_request dynamically generates an <iframe>which loads the URL http://www.facebook.com/beacon/auth_iframe.php and passes the query_params. At this point, Facebook now knows about the news feed item whether you choose to publish it or not. 

When you read this you realize just how insidious the problem actually is. Facebook isn't simply learning about every action taken by Facebook users on affiliate sites, it is learning about every action taken by every user of these affiliate sites regardless of whether they are Facebook users or not.

At first I assumed that the affiliates sites would call some sort of IsFacebookUser() API and then decide whether to send the action or not. Of course, this is still broken since the affiliate site has told Facebook that you are a user of the site, and depending on the return value of the hypothetical function the affiliate in turn learns that you are a Facebook user.

But no, it is actually worse than that. The affiliate sites are pretty much dumping their entire customer database into Facebook's lap, FOR FREE and without their customers permission. What. The. Fuck.

The icing on the cake is the following excerpt from the Facebook Beacon page

Stories of a user's engagement with your site may be displayed in his or her profile and in News Feed. These stories will act as a word-of-mouth promotion for your business and may be seen by friends who are also likely to be interested in your product. You can increase the number of friends who see these stories with Facebook Social Ads.

So after giving Facebook millions of dollars in customer intelligence for free in exchange for spamming their users, Facebook doesn't even guarantee their affiliates that the spam will even get sent. Instead these sites have to pay Facebook to "increase the chances" that they get some return for the free customer intelligence they just gave Facebook.

This reminds me of the story of Tom Sawyer tricking people into paying him to paint a fence he was supposed to paint as part of his chores.

At the end of the day, Facebook can't fix the privacy problems I mentioned in my previous post in a way that completely preserves their users privacy without completely changing the design and implementation of Facebook Beacon. Until then, we'll likely see more misdirection, more red herrings and more violations of user privacy to make a quick buck. 


 

November 27, 2007
@ 04:00 AM

Recently I’ve read a number of negative posts about the Facebook Beacon which highlight how easy it is for a company to completely misjudge the privacy implications and ramifications of certain features in social software applications.

Charlene Li, a Principal Analyst at Forrester Research who specializing in social software trends and marketing, writes in her blog post Close encounter with Facebook Beacon  

I put a lot of trust in sites like Facebook to do the right thing when it comes to privacy. After all, the only stuff that gets out into the public is the stuff that I actually put in. Until now.

Earlier this week, I bought a coffee table on Overstock.com. When I next logged into Facebook and saw this at the top of my newsfeed:

I was pretty surprised to see this, because I received no notification while I was on Overstock.com that they had the Facebook Beacon installed on the site. If they had, I would have turned it off.

I used my personal email address to buy the coffee table, so I was puzzled why and how this "personal" activity was being associated with my "public" Facebook profile.

David Treadwell, a corporate vice president of Windows Live, writes in his blog post entitled Blockbuster, you're fired

Yesterday evening, I decided to add a few movies to my Blockbuster queue. Upon adding movies, I was surprised to see toasts from Facebook showing up on the Blockbuster site indicating that something was being added to my Facebook news feed. When I finished adding movies, I went to Facebook to see what was going on. I was then quite surprised to learn that Blockbuster and Facebook were conspiring to broadcast my movie selections to my Facebook friends.

I am not normally uptight about privacy issues, but you guys really crossed the line on this one:

  • I had never told either Blockbuster or Facebook that you should share my movie selections with friends.
  • Neither of you asked me if you could take this action. You just went ahead and did it, assuming that I would not mind.
  • This sharing of information about me without my informed consent about the mechanism of sharing is absolutely unacceptable to me.

You can find similar complaints all over the Web from similarly Web savvy folks who you typically don’t see griping about privacy issues. In all of the complaints raised, the underlying theme is that Facebook violated the principle of putting the user in control of their user experience.

As someone who works on a competing service I have to give the folks on Facebook credit for shipping the Facebook Beacon so quickly. I assumed something like that was still about six months away from being on their radar. I do give them poor marks when it comes to how this feature has been rolled out. There are several problems with how this feature has been rolled out when it comes to how it affects their users.

  1. Linking identities and data sharing without user permission: One of the thinks people have found creepy about this feature is that they are automatically discovered to be Facebook users on sites that they have not told they use Facebook. In Charlene’s case, she actually uses different email addresses to log in on both sites which must have seemed even doubly weird to her at first. As Ethan Zuckerman points out in his post Facebook changes the norms for web purchasing and privacy this completely upturns user expectations of how privacy on the Web works especially when it comes to cookies.  

    It's a genuine concern that Facebook has opened a Pandora's box when you consider what could happen if it is deemed socially acceptable for Web sites to use cookies to actively identify users across sites as opposed to the passive way it is done today. I’m sure the folks at Google would be excited about this since thanks to AdSense and DoubleClick, they  probably have cookies on every computer on the Web that has cookies turned enabled in the Web browser. Today it’s Facebook, tomorrow Amazon and eBay are posting your purchase history to every OpenSocial enabled web site courtesy of the cookies from these sites or from Google ads on your machine.

  2. No global opt-out: There is no way to turn off this feature. The best you get is that when a site tries to publish an update to your news feed and mini-feed, you get an entry for the site added to your Privacy Settings for External Websites page on Facebook. I guess it never occured to Mark Zuckerburg and Justin Rosenstein that not sharing my purchase history with Facebook is a valid privacy option. Why do I have to police this list and refer back to it every couple of days to figure out if some new Web site is now publishing my private data to Facebook without my permission? 

    I expect that kind of myopia and hubris from the Googles and Microsofts of the world not Facebook. Wow, the honeymoon lasted shorter than I expected.

I suspect that Facebook will loathe fixing both issues. The first issue can’t really be solved by having partner sites provide an opt-in mechanism because there is the valid concern that (i) people won’t opt-in to the feature and (ii) the experience and messaging will vary too much from site to site for users to have a consistent set of expectations. This then points to Facebook having an opt-in page for partner sites that is part of the Facebook settings page for this feature but that may start getting away from the add 3 lines of code to reach millions of users sales pitch which they have going. Adding a global opt-out button is also similarly fraught with down side for Facebook.

At this point, they’ll have to do something. I’ll be impressed if they address both issues. Anything less is simply not good enough.

PS: The technically inclined folks in the audience should take a look at Jay Goldman’s excellent Deconstruction of the Facebook Beacon Javascript. Found via Sam Ruby.

Now playing: Eightball & MJG - Relax & Take Notes (feat. Project Pat & Notorious B.I.G.)


 

I've been pondering the implications of Facebook's SocialAds announcement and it has created some interesting lines of thought. The moment the pin dropped was when Dave Winer linked to one of his old posts that contains the following money quote

that's when the whole idea of advertising will go poof, will disappear. If it's perfectly targeted, it isn't advertising, it's information. Information is welcome, advertising is offensive.

If you think about it, the reason Google makes so much money from search advertising is because the ads are particularly relevant when a user is seeking information or a trusted opinion as part of the process to make a commercial decision. If I'm searching for "iPod" or "car insurance" then it is quite likely that ads selling me these products are relevant to my search and are actually considered to be information instead of intrusive advertising

Where Google's model breaks down is that a large amount of the advertising out there is intended to make you want to buy crap that you weren't even interested in until you saw the ads. In addition, trusted recommendations are a powerful way to convince customers to make purchases they were otherwise not considering. Former Amazon employee Greg Linden has written blog posts that indicate that 20% - 35% of Amazon's sales comes from recommendations like "people who like 50 Cent also like G-Unit". Given that Amazon made over 10 billion dollars in revenue last year (see financials), this means that $2 billion to $3.5 billion of that revenue is based on what Facebook is calling "social" ads.

So what does all this have to do with the title of my blog post? Glad you asked. Recently Yaron and I were chatting about the virtues of the Facebook platform. He argued that the fact that applications are encouraged to keep their data within their own silos (e.g. Flixster isn't supposed to be mucking with my iLike data and vice versa) prevents everyone [including Facebook] from benefiting from all this profile data being created from alternate sources. I argued that seeing the complexities introduced by having multiple applications being able to write to the same data store (e.g. the Windows registry) it's a lot better for users and app developers if they don't have to worry that some half baked app written by some drunken college kid is going to hose their Scrabulous scores or corrupt all their movie ratings. 

However what this means is that some of the juiciest data to serve "social" ads against within Facebook (i.e. movies and music) is not in Facebook's databases but in the databases of the developers of Facebook applications like Slide, iLike and Flixster. Considering the following entry that shows up in my friends news feeds after I performed an action in iLike ,

This entry could be improved with "social" ads in a way that is informative and valuable to my friends while also providing financial value to the application developer. For instance, would you consider the following changes to that entry to be advertising or information?

Flixster does an even worse job than iLike in making the actions they show in my news feed to be both useful and monetizable. Here's the kind of stuff that shows up in my news feed from Flixster

I don't know about you but I consider this spam. In fact, it is also misleading since what it really means is that someone on my friends list (Steve Gordon) has also installed the Flixster application on their profile. However what if the application actually published some of my movie ratings into the news feed with more context such as

People keep asking how Facebook application developers will make money. From where I'm sitting, this looks like a freaking gold mine. The problem seems to be that these applications either haven't yet figured out how lucrative a position they're in or are still in the audience acquisition phase until they flip to the highest bidder.

If Mark Zuckerburg has any strategic bone in his body, he'd snap up these companies before a hostile competitor like Google or Fox Interactive Media does. I'd put money on it that people are slowly realizing this all over Silicon Valley. 

What do you think?


 

I've seen a couple of recent articles talking about how Facebook has turned on it's platform developers with it's most recent announcements. Fortune magazine has an article today entitled Fear Among Facebook Developers which states

Zuckerberg wouldn’t deny it. On stage at the Web 2.0 conference in October in San Francisco, he acknowledged that his company reserves the right to build anything it wants and compete with any of its independent developers, but that the company intends to do this fairly. “We need to make sure we have the flexibility to do what we need as the platform grows—to be flexible enough able to add in the next big component [like the News Feed],” he said.

Yesterday Erick Schonfeld wrote an article on TechCrunch entitled iLike vs. Facebook: The Battle For The Music Artist that contains the following excerpt

Instead, Facebook is treating music artists just like any other brands, which can also set up their own Facebook pages, collect fans, and market to them directly. Yet, when it comes to music artists, one of Facebook’s most popular application developers, iLike, is doing the exact same thing.
...
So if you are a music artist, you now have to make a decision: Do you go with the iLike page as your main Facebook page (and take advantage of the nearly 10 million members who use the iLike app), or do you go with your own advertiser page on Facebook? Case in point: the new Facebook page for 50 Cent (shown left) had only three fans when it first went up just after midnight, compared to 1.2 million fans on his iLike page on Facebook.

This is a tale as old as the hills. Software platforms evolve and often this means incorporating features that were once considered as "features to be provided by others" as core parts of the platform. There are thousands of examples of application developers adding value to a platform that eventually became features of the platform due to popular demand. Whether it is adding a TCP/IP stack to the operating system, tabbed browsing to a Web browser or adding persistent searches to a Web mail application, it's all the same story. It is hard to argue that it isn't better for users such functionality to be a native part of the platform or underlying application, however it often leaves the platform developers in a lurch.

If the application developer cannot find a new way to add value to the platform then their usefulness to users comes to an end. This doesn't make it a slam dunk that once the platform vendor sees the value added by an application on it's platform, that things will eventually go sour for the application. There are many examples of vendors trying to compete with an application on their platform only to concede defeat and then try to acquire the company; PhotoBucket's acquisition by MySpace and Oracle's attempt to acquire BEA are two recent examples. [Editors note - I suspect that iLike vs. Facebook will end up going the same route as well]. In other cases, entry into the application space by the platform vendor helps to validate the market and draws more visibility to it from users.  

At the end of the day, articles like the ones I've mentioned above serve to prove that Facebook has actually built a viable and successful platform given that it is following the patterns of other successful platforms from the past several decades of the software industry.


 

Mitch Radcliffe has a blog post on ZDNet entitled Google: Does it have to be all FUD all the time? where he writes

Fear, uncertainty and doubt (FUD–see Wikipedia) is how IBM tried to retain its dominance, what Microsoft used to cement its monopoly and, now, I suggest we review recent Google news and wonder:
...
Basically, four of the last five press releases from Google have amounted to “me and my friends are going to…” beat a major competitor or rule a marketplace based on pre-announcements without a great deal of substance or products that can be seen and used today. Google sounds more like Microsoft circa 1988-to-1992, when it was launching consortia right and left to block competitors without delivering much, or any, real product. Microsoft still does this, but it doesn’t enjoy the credibility (or, better, the credulousness) that greet Google announcements.

It seems I'm not the only one that noticed this trend. I guess the Features Not Products initiative morphed into a "FUD Not Products" initiative somewhere along the line. :)

Or maybe they just hit a critical mass of ex-Microsoft employees from the old days.

PS: For bonus points, read Dave Winer's Why Google launched OpenSocial.


 

Disclaimer: This post does not reflect the opinions, thoughts, strategies or future intentions of my employer. These are solely my personal opinions. If you are seeking official position statements from Microsoft, please go here.

One of the Google folks working on OpenSocial sent me a message via Facebook asking what I thought about the technical details of the recent announcements. Since my day job is working on social networking platforms for Web properties at Microsoft and I'm deeply interested in RESTful protocols, this is something I definitely have some thoughts about. Below is what started off as a private message but ended up being long enough to be it's own blog post.

First Impressions

In reading the OpenSocial API documentation it seems clear that is intended to be the functional equivalent of the Facebook platform. Instead of the Facebook users and friends APIs, we get the OpenSocial People and Friends Data API. Instead of the Facebook feed API, we get the OpenSocial Activities API. Instead of the Facebook Data Store API, we get the OpenSocial Persistence Data API. Instead of FQL as a friendly alternative to the various REST APIs we get a JavaScript object model.  

In general, I personally prefer the Facebook platform to OpenSocial. This is due to three reasons

  • There is no alternative to the deep integration into the Web site's user experience that is facilitated with FBML.  
  • I prefer idiomatic XML to tunnelling data through Atom feeds in ways that [in my opinion] add unnecessary cruft.
  • The Facebook APIs encourage developers to build social and item relationship graphs within their application while the OpenSocial seems only concerned with developers stuffing data in key/value pairs.

The Javascript API

At first I assumed the OpenSocial JavaScript API would provide similar functionality to FBML given the large number of sound bites quoting Google employees stating that instead of "proprietary markup" you could use "standard JavaScript" to build OpenSocial applications. However it seems the JavaScript API is simply a wrapper on top of the various REST APIs. I'm sure there's some comment one could make questioning if REST APIs are so simple why do developers feel the need to hide them behind object models?

Given the varying features and user interface choices in social networking sites, it is unsurprising that there is no rich mechanism specified for adding entry points to the application into the container sites user interface. However it is surprising that no user interface hooks are specified at all. This is surprising given that there are some common metaphors in social networking sites (e.g. a profile page, a friends list, etc) which can be interacted with in a standard way.  It is also shocking that Google attacked Facebook's use of "proprietary markup" only to not even ship an equivalent feature.

The People and Friends Data API 

The People and Friends Data API is used to retrieve information about a user or the user's friends as an Atom feed. Each user is represented as an atom:entry which is a PersonKind (which should not be confused with an Atom person construct). It is expected that the URL structure for accessing people and friends feeds will be of the form  http://<domain>/feeds/people/<userid> and http://<domain>/feeds/people/<userid>/friends respectively.

Compare the following response to a request for a user's information using OpenSocial with the equivalent Facebook API call response.

GET http://orkut.com/feeds/people/14358878523263729569
<entry xmlns='http://www.w3.org/2005/Atom' xmlns:georss='http://www.georss.org/georss' xmlns:gd='http://schemas.google.com/g/2005'>
<id>http://sandbox.orkut.com:80/feeds/people/14358878523263729569</id>
<updated>2007-10-28T14:01:29.948-07:00</updated>
<title>Elizabeth Bennet</title>
<link rel='thumbnail' type='image/*' href='http://img1.orkut.com/images/small/1193601584/115566312.jpg'/>
<link rel='alternate' type='text/html' href='http://orkut.com/Profile.aspx?uid=17583631990196664929'/>
<link rel='self' type='application/atom+xml' href='http://sandbox.orkut.com:80/feeds/people/14358878523263729569'/>
<georss:where>
<gml:Point xmlns:gml='http://www.opengis.net/gml'>
<gml:pos>51.668674 -0.066235</gml:pos>
</gml:Point>
</georss:where>
<gd:extendedProperty name='lang' value='en-US'/>
<gd:postalAddress/>
</entry>

Below is the what the above information would look like if returned by Facebook's users.getInfo method

GET

<users_getInfo_response xmlns="http://api.facebook.com/1.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://api.facebook.com/1.0/ http://api.facebook.com/1.0/facebook.xsd" list="true">
<user>
<uid>14358878523263729569</uid>
<current_location>
<city>Palo Alto</city>
<state>CA</state>
<country>United States</country>
<zip>94303</zip>
</current_location>
<first_name>Elizabeth</first_name>
<is_app_user>1</is_app_user>
<has_added_app>1</has_added_app>
<pic>http://photos-055.facebook.com/ip007/profile3/1271/65/s8055_39735.jpg</pic>
</user>
</users_getInfo_response>

I've already mentioned that I prefer idiomatic XML to tunnelling data through Atom feeds. Comparing the readability of both examples should explain why.

The Activities Data API 

A number of social networking sites now provide a feature which enables users to see the recent activities of members of their social network in an activity stream. The Facebook news feed, Orkut's updates from your friends, and the Windows Live Spaces what's new page are all examples of this feature. The OpenSocial Activities Data API provides a mechanism for OpenSocial applications to access and update this activity stream as an Atom feed. All of the users activities or all activities from a specific application can be accessed using URIs of the form  http://<domain>/activities/feeds/activities/user/<userID> and http://<domain>/activities/feeds/activities/user/<userID>/source/<sourceID> respectively.  

Currently there is no reference documentation on this API. My assumption is that since Orkut is the only OpenSocial site that supports this feature, it is difficult to produce a spec that will work for other services without it being a verbatim description of Orkut's implementation.

There are some notes on how Orkut attempts to prevents applications from spamming a user's activity stream. For one, applications are only allowed to update the activity stream for their source directly instead of the activity stream for the user. I assume that Google applies some filter to the union of all the source specific activity streams before generating the user's activity feed to eliminate spam. Secondly, applications are monitored to see if they post too many messages to the activity stream or if they post promotional messages instead of the user's activities to the stream. All of this makes it seem difficult to see how one could specify the behavior of this API and feature set reliably for a diverse set of social networking sites.

The Persistence Data API 

The OpenSocial Persistence API allows applications to store and retrieve key<->value pairs that are either user-specific or are global to the application. An example of the former is a listing of company name and stock ticker pairs while an example of the latter is a user's stock portfolio. The feed of global key<->value pairs for an application can be accessed at a URL of the form http://<domain>/feeds/apps/<appID>/persistence/global for the entire feed and http://<domain>/feeds/apps/<appID>/persistence/global/<key> if seeking a particular key<->value pair. User-specific key<->value pairs are available at the URL of the form http://<domain>/feeds/apps/<appID>/persistence/<userID>/instance/<instanceID>.

This is probably the least interesting aspect of the API. A simple persistence API like this is useful for applications with simple storage needs that need to store user preferences or simple textual data that is needed by the application. However you aren't going to use this as the data storage platform for applications like iLike, Flixster or Scrabulous.

However I will add that an Atom feed seems like a horrible representation for a list of key<->value pairs. It's so bad that the documentation doesn't provide an example of such a feed.

Hosting OpenSocial Applications

The documentation on hosting OpenSocial applications implies that any site that can host Google gadgets can also host OpenSocial applications. In practice, it means that any site that you can place a <script> element on can point to a gadget and thus render it. Whether the application will actually work will depend on whether the hosting service has actually implemented the OpenSocial Service Provider Interface (SPI).

Unfortunately, the documentation on implementing the OpenSocial SPI is missing in action. From the Google site

To host OpenSocial apps, your website must support the SPI side of the OpenSocial APIs. Usually your SPI will connect to your own social network, so that an OpenSocial app added to your website automatically uses your site's data. However, it is possible to use data from another social network as well, should you prefer. Soon, we will provide a development kit with documentation and code to better support OpenSocial websites, along with a sample sandbox which implements the OpenSocial SPI using in-memory storage. The SPI implements:

  • Adding and removing friends
  • Adding and removing apps
  • Storing activities
  • Retrieving activity streams for self and friends
  • Storing and retrieving per-app and per-app-per-user data

The OpenSocial website development kit will include full SPI documentation. It will provide open source reference implementations for both client and server components.

I assume that the meat of the OpenSocial SPI is documentation is just more detailed rules about how to implement the REST APIs described above. The interesting bits will likely be the reference implementations of the API which will likely become the de facto standard implementations instead of encouraging dozens of buggy incompatible versions of the OpenSocial API to bloom.   

Conclusion

In general I believe that any effort to standardize the widget/gadget APIs exposed by various social networking sites and AJAX homepages (e.g. iGoogle, Netvibes, Live.com, etc) is a good thing. Niall Kennedy has an excellent series of articles on Web Widget formats and Web Widget update technologies that shows how diverse and disparate the technologies that developers have to learn and utilize when they want to build widgets for various sites. Given that Web widgets are now a known quantity, the time is ripe for some standardization.

That said, there are a number of things that give me cause to pause with regards to OpenSocial

  1. A common practice in the software industry today is to prefix "Open" to the name of your technology which automatically gives it an aura of goodness while attempting to paint competing technologies as being evil and "closed". Examples include OpenDocument, OpenID, OpenXML, OAuth, etc. In this case, OpenSocial is being positioned as an "open" alternative to the Facebook platform.  However as bloggers like Shelley Powers, Danny Ayers and Russell Beattie have pointed out, there isn't much "open" about OpenSocial. Russell Beattie asks in his post Where the hell is the Container API?

    Would people be jumping on this bandwagon so readily if it was Microsoft unilaterally coming up with an API, holding secret meetings geared towards undercutting the market leader, and then making sure that only those anointed partners get a head start on launch day by making sure a key part of the API isn't released - even in alpha. (It obviously exists already, all the partners have that spec and even sample code, I'm sure. The rest of us don't get access yet, until the GOOG says otherwise).

    Let's say we ignore that the process for creating the technology was not "open" nor have key aspects of the technology even been unveiled [which makes this more of a FUD announcement to take the wind out of Facebook's sails than an actual technology announcement], is the technology itself open? Shelley Powers points out her post Terms that

    Perhaps the world will read the terms of use of the API, and realize this is not an open API; this is a free API, owned and controlled by one company only: Google. Hopefully, the world will remember another time when Google offered a free API and then pulled it. Maybe the world will also take a deeper look and realize that the functionality is dependent on Google hosted technology, which has its own terms of service (including adding ads at the discretion of Google), and that building an OpenSocial application ties Google into your application, and Google into every social networking site that buys into the Dream.

    Google has announced a technology platform that is every bit as proprietary as Facebook's. The only difference is that they've cut deals with some companies to utilize their proprietary platform while Facebook's platform is only for use on the Facebook site. If Zuckerburg announces next week that the Facebook platform is freely implementable by any 3rd party Web site, where does that leave OpenSocial? After all, the Facebook platform is actually a proven, working system with complete documentation instead of the incomplete rush job that OpenSocial clearly is right now.

    There are all sorts of forums for proposing and discussing open Web technologies including the IETF, W3C, OASIS and even ECMA. Until all of the underlying technologies in OpenSocial have been handed over to one or more of these standards bodies, this is a case of the proprietary pot calling the proprietary kettle black.

  2. One of the things that comes along with OpenSocial is that Google has now proposed GData as the standard protocol for interacting with social graphs on the Web. This is something that I've been worried about for a while and I've written a couple of blog posts to address this topic because it is not clear that the Atom Publishing Protocol upon which GData is based works well outside it's original purpise of editing blog posts and the like. I'm not the only one that feels this way.

    Danny Ayers wrote in his post Open? Social?

    However the People Data API is cruel and unusual. It first stretches Atom until it creaks with "each entry in the People or Friends feed is a PersonKind"; then gives a further tug  (a person's name is represented using atom:title) then extends it even more (a person's email is gd:email) and finally mops up all the blood, sweat and dribble:

    Key value parameters - gd:extendedProperty - "As different social networks and other sources of People data have many different named fields, this provides a way for them to be passed on generally. Agreeing on common naming conventions is to be decided in future."

    Got to admire the attempt, but (to mix the metaphorical namespaces) silk purses don't make very good sow's ears either.

    In addition, AtomPub geek extraordinairre, Tim Bray wrote in his blog post entitled Web3S

    If you decide you totally can’t model your world as collections of entries populated with hyperlinks to express relationships, well then I guess APP’s not for you. And at the level of engineering intuition, I have to say that a monster online address book does feel different at a deep level from most online “publications” (I thought that was why we had LDAP... but I repeat myself).

    Now that we have AtomPub/GData as a de facto standard protocol for accessing various kinds of non-microcontent data on the Web as a reality, I'm done debating its suitability for the task since the horse has already left the barn. However I will continue to ask when will GData be RFC 5023 compliant?

  3. At the end of the day, the most disappointing thing about OpenSocial is that it doesn't really further the conversation about actual interoperability across social networking sites. If I use Orkut, I still need a MySpace account to interact with my friends on that site. Some people have claimed that OpenSocial will enable routing around such lock-in via applications like iLike and Flixster which have their own social networks and thus could build cross-site social networking services since they will be hosted on multiple social networking sites. However the tough part of this problem is how a hosted application knows that carnage4life@windowslivespaces is the same user as DareObasanjo@Facebook? It seems OpenSocial completely punts on satisfying this scenario even though it wouldn't be hard to add this as a requirement of the system. I guess the various applications can create their own user account systems and then do the cross-site social network bridging that way, which sucks because it will be a lot of duplicative work and will require users to create even more accounts with various services.

    Given that the big widget vendors like iLike, Slide and RockYou already have their users creating accounts on their sites that can be tied back to which social networking site the user utilizes their widgets on, this might be a moot point. Wouldn't it be mad cool if the Top Friends Facebook application could also show your top friends from MySpace or Orkut? I suspect the valuation of various widget companies will be revised upwards in the coming months.

  4. There is no mention of a user-centric application authorization model. Specifically, there is no discussion of how users grant and revoke permission to access their personal data to various OpenSocial applications. Regular readers of my blog are familiar with my mantra of putting the user in control which is why I've been so enthusiastic about OAuth. Although there is some mention of Google's Authentication for Web Application in the documentation, this seems specific to Google's implementation of OpenSocial hosting and it is unclear to me that we should expect that this is the same model that will be utilized by MySpace, Bebo, TypePad or any of the other social networking sites that have promised to implement OpenSocial. On the other hand, Facebook has a well thought out applications permission model and I would have thought it would be quite easy to simply reverse engineer that and add it to the OpenSocial spec than to simply punt on this problem.

Despite these misgivings, I think this is a step in the right direction. Web widget and social graph APIs need to be standardized across the Web.

PS: I've subscribed to the Google OpenSocial blog. So far there have only been posts by clueless marketing types but I'm sure interesting technical information that addresses some of the points above will be forthcoming.


 

In a post entitled Checkmate? MySpace, Bebo and SixApart To Join Google OpenSocial (confirmed) Mike Arrington writes

Google may have just come out of nowhere and checkmated Facebook in the social networking power struggle.

Update (12:30 PST): On a press call with Google now. This was embargoed for 5:30 pm PST but they’ve moved the time up to 12:30 PST (now). Press release will go out later this evening. My notes:

On the call, Google CEO Eric Schmidt said “we’ve been working with MySpace for more than a year in secret on this” (likely corresponding to their advertising deal announced a year ago).

MySpace says their new platform efforts will be entirely focused on OpenSocial.

The press release names Engage.com, Friendster, hi5, Hyves, imeem, LinkedIn, Ning, Oracle, orkut, Plaxo, Salesforce.com, Six Apart, Tianji, Viadeo, and XING as current OpenSocial partners.

We’re seeing a Flixster application on MySpace now through the OpenSocial APIs. Flixster says it took them less than a day to create this. I’ll add screen shots below.

Here’s the big question - Will Facebook now be forced to join OpenSocial? Google says they are talking to “everyone.” This is a major strategic decision for Facebook, and they may have little choice but to join this coalition.

Bebo has also joined OpenSocial.

I'm confused as to how Mike Arrington considers this a checkmate by Google. At the end of the day, this announcement is simply that folks like Slide and RockYou don't have to maintain multiple code bases for their widgets on various popular social networking sites. In addition, it brings the widget/gadget platform on these sites to a similar level to the Facebook platform. Of course, it won’t be on the same level unless it meets all the criteria from my post on how developers should evaluate the MySpace platform. Which is unlikely since besides MySpace, none of those sites have the userbase or engagement of Facebook users nor does any of them have the same kind of viral properties in distributing applications that Facebook platform has built-in

At the end of the day, will we see widget developers like the folks at iLike, Slide or Scrabulous leave the Facebook platform because of these announcements? Unlikely.

Will we see a mass migration from Facebook to MySpace or Orkut because you can now add Flixster or Scrabulous to your profile on these sites? Probably not.

So how is this a checkmate again?

OpenSocial simply keeps Facebook’s competitors in the game. It is more like a successful kingside castle than a checkmate.

Now playing: Backstreet Boys - Incomplete


 

In thinking about the Google OpenSocial Announcement I realized how much some of Google's recent moves remind me of Microsoft of old [for some undeclared definition of old]. We have

  • Google is now a two trick pony with two self reinforcing monopolies (Search + AdSense/Adwords) which is reminiscent of the self reinforcing nature of Microsoft's Windows + Office.

  • A scared overreaction to the growing hype of a startup that is eclipsing it in coolness. See OpenSocial reaction to Facebook

  • Several high level and very visible defections to said startup from people who were highly regarded there. Seems like just yesterday we were reading similar articles and blog posts about MSFT -> Google employee defections.

  • Said startup may be attacking Google's core monopoly in ways that were unforeseen but make sense in hindsight. See Facebook's SocialAds announcements.
  • Creation of de jure and de facto industry standards which are pitched as being great for customers but seem suspiciously like attempts at Fire & Motion. Compare WS-* to AtomPub + GData + OAuth + OpenSocial + ???.


 

There’s nothing like a successful company with a near monopoly to force the software industry to come up with standards. Or in this case, as in many others, force it’s competitors to band together and call what they are doing the standard because more than one vendor supports it.

From TechCrunch’s article Details Revealed: Google OpenSocial(To Launch Thursday we learn

Google wants to create an easy way for developers to create an application that works on all social networks. And if they pull it off, they’ll be in the center, controlling the network.

What They’re Launching

OpenSocial is a set of three common APIs, defined by Google with input from partners, that allow developers to access core functions and information at social networks:

  • Profile Information (user data)
  • Friends Information (social graph)
  • Activities (things that happen, News Feed type stuff)

Hosts agree to accept the API calls and return appropriate data. Google won’t try to provide universal API coverage for special use cases, instead focusing on the most common uses. Specialized functions/data can be accessed from the hosts directly via their own APIs.

Unlike Facebook, OpenSocial does not have its own markup language (Facebook requires use of FBML for security reasons, but it also makes code unusable outside of Facebook). Instead, developers use normal javascript and html (and can embed Flash elements). The benefit of the Google approach is that developers can use much of their existing front end code and simply tailor it slightly for OpenSocial, so creating applications is even easier than on Facebook.

Similar details are available from folks like Om Malik and Marc Andreesen

This is a brilliant move. I’ve blogged on multiple occassions that the disparate widget platforms in social networking sites is a burden for widget developers and will lead to a “winner takes all” situation because no one wants to support umpteen different platforms. If enough momentum gains around OpenSocial, then three things will happen

  • Widget developers will start to favor coding to OpenSocial because it supports multiple sites as well as targeting the Facebook platform  
  • Eventually Facebook platform developers will start asking Zuckerburg and company to support OpenSocial so they only need to worry about one code base (kinda, it won’t be that easy)  
  • Other companies with proprietary widget platforms or plans to create one will bow down to the tide and adopt OpenSocial

Of course, this requires a popular social networking site with a wide audience (e.g. MySpace) to adopt the platform before we see this kind of traction.

However this is the only thing Google could have done that makes any sense. Building a clone of the Facebook platform like some social networking sites planned would have been dumb because that would be the tail wagging the dog. Similarly building a competing proprietary platform would also have been dumb due to the winner takes all problem I mentioned earlier.

This is the only move that has a chance of actually giving their anti-Facebook platform a chance of being successful.

I wonder how my coworkers in Windows Live are going to take this news?

Now playing: 50 Cent - All Of Me (Feat. Mary J Blige) (Prod by Jake One)


 

Web companies are primarily media companies. Their job is to assemble audiences and then sell ads to their captive audiences. This is what the newspaper, television and magazine industries did before them but for some reason Web companies like Google love to claim that they aren’t media companies.  The irony is that Google stumbled on the most lucrative media property of all time (Web search results) as well as the most lucrative way of monetizing it (auction based keyword sales).

Web 1.0 was about gathering a bunch of eyeballs and then entertaining them with a bunch of content you created. Web 2.0 is all about gathering a bunch of eyeballs then having them entertain themselves with content they’ve created (also known as UGC). The most important thing hasn’t changed, it’s about gathering eyeballs.

This takes us to social networking sites like MySpace and Facebook which have taken the act of gathering eyeballs and then having your users entertain themselves with the help of random developers and elevated it to a fine art.  Things get interesting when you realize that a lot of these “random developers” have built compelling experiences and have gathered a significant audience in their own right on the backs of these social networking sites.

In a blog post entitled Scrabulous Jason Antonelli writes

I've become addicted to Scrabulous on Facebook…I'm on my 3rd game of Scrabble with my mom (yes I won two games so far) - and it usually starts with a phone call where we play for a bit and chat about various mom/son things.  My mom lives 3000+ miles away -- she's in Wilmington, NC and I'm in Seattle.  After a while when one of us needs to go do something else - we'll go ahead and play it throughout the day - maybe each making a 1-2 moves a day.

This is so much fun (though I wonder how much it would be if I didn't have a 100% win rate) and it clearly is bringing social networks into the mainstream.  The fact that this game is hosted on the Facebook platform (although it does look like it has a standalone site as well) gave two major wins to Facebook.  First I actually convinced by mom (I'm 33 so you can only guess her age!) to signup for Facebook - the first person I got to sign up to any social networking application even including the one I've worked on.  And second, both my mom and I log in to Facebook multiple times a day.

I’m similarly hooked on Scrabulous on Facebook and estimate that 75% of my page views to *.facebook.com are actually to the Scrabulous application page. I’m definitely see more ads hosted within Scrabulous than I see those hosted within Facebook*. Which leads me to wonder especially when I notice that Scrabulous now sports an ads by Google logo. I specifically wondered about the reach of the most popular widget makers and was surprised when I took a look at the Slide.com About Page which informed me that  

Slide is the largest personal media network in the world, reaching more than 134 million unique global viewers each month and 30 percent of the U.S. Internet audience. We help people express themselves and tell stories through personalized photos and videos created on Slide.com and viewed anywhere on the web or desktop.

Slide widgets — including Slideshows, Guestbooks, SkinFlix and FunPix — are popular on top social networking and blog platforms, including MySpace, Facebook, Bebo, Hi5, Friendster, Tagged, Piczo and Blogger. Slide is also the leading application developer on Facebook with more than 63 million applications installed, including SuperPoke and Top Friends, the most active application by more than four times that of any other 3rd party developer.

Whoa. I suspect that once MySpace ships their platform which relaxes the rules on advertising within widgets, companies like Slide will start to look like very interesting acquisition targets. Or maybe there’s enough money to be made generating the majority of the page views on the most popular social networking sites in the world, that there’ll be no incentive to cash out. Interesting times ahead.

* I do think I’ve clicked more on Facebook hosted ads than those hosted within Scrabulous so that may not be a useful metric. Smile

Now playing: Fabolous - This Is My Party


 

Disclaimer: Although I work on the What’s New feed in Windows Live Spaces this should not be considered an announcement or precursor to an announcement of upcoming features of any Windows Live service.

I spend a lot of my time these days thinking about digital lifestyle aggregators such as Facebook and FriendFeed. One of the things I wonder about is how to make them more relevant to users as a way to stay connected to each other without seeming confusing, overwhelming or just plain spammy.

For instance, I look at the Facebook News Feed as the first significant implementation of this concept to hit the mainstream and I try to see what we can learn from their mistakes and where there is room for improvement. Below are two mistakes and one place I see room for improvement in the news feed as currently implemented by Facebook. The screenshot below is provided as a reference point.

facebook news feed

Wall Posts

I’m now quite convinced that having wall posts show up in the news feed is a mistake. In general, Facebook already indulges in bad design by having a Wall-to-Wall posting which means that you can be viewing a friend’s wall and may only see one half of the conversation. So there is always a confusing loss of context when reading a wall on Facebook. This loss of context is exacerbated by adding wall posts to the news feed since now we not only have to deal with hearing one side of a conversation. Instead a user logs in and is confronted with a statement from the middle of a conversation, clicks through and only sees half the conversation, tries to click through to that and may not have access to both user’s walls.

More than once I’ve logged into Facebook and been confronted with wall posts that would have been embarassing to the posters if they realized that their banter on some person’s guestbook wall was being broadcast out of context to all their co-workers, their manager and even their VP/CxO via the news feed.

Application Installations

Although I don’t agree with Kara Swisher for criticizing Facebook applications as being mostly trivial time wasters instead of professional tools, I do agree with her that most apps on the site aren’t of value. This means that using up my screen real estate to tell me that a buddy has installed the Pink Ribbon application or the ProductPulse application is spam almost every single time you do it. One should also consider that Facebook limits the amount of updates from your friends they show in your news feed to ensure a good mix of updates. I suspect most users would gladly trade the slots taken up by notifications of application installs for more personally relevant updates from their social network.

The notifications about application installs showing up in the news feed is nice for developers but I question it’s value to users. Especially when you consider that Facebook applications already have ways to spread virally via application requests which has unfortunately led to Facebook Application Fatigue by their users.

Groups and Events

Today, a lot of groups on Facebook are there primarily as a way to declare affiliation as opposed to being an active community of users such as you’d find on sites like MSN Groups or Yahoo! Groups. I joined groups like I Dont care How Comfortable Crocs Are, You Look Like A Dumbass, I Am Fluent in Sarcasm, and If Wikipedia Says It, It Must Be True because I thought joining them would look funny on my friend’s news feeds as opposed to wanting to be part of these groups either as a lurker or as a regular discussion participant.

One question is why Facebook uses the news feed to drive user to user traffic but not user to group traffic besides the “Dare has joined People Who Always Have To Spell Their Names For Other People” which encourages people to join the group but doesn’t encourage them to participate in the group. It may be that they don’t want users creating online groups within the site like you find in services like MSN Groups or Yahoo! Groups or it could just be that their platform can’t support that scale of activity. I wonder…

The same questions apply for events as well. It would be cool if after I was invited to an event, I also got news feed updates via my news feed later on telling me if my friends were attending or that the event was getting lots of attendees which may influence my attendance. Again, you have to wonder why such obvious enhancements haven’t made it into their service.

Now playing: Three 6 Mafia - Late Nite Tip


 

Erick Schonfeld from TechCrunch writes in his post Windows Live SkyDrive Doubles Storage to 1GB, Still Can’t Keep Up With Gmail that

Microsoft doubled the online storage consumers can get for free in Windows Live SkyDrive. It’s hard to get excited about that when Gmail is already giving me 2.9 GB of storage, with more on the way—4GB by the end of the month, and 6GB by early January, according to one estimate.

You’d think that someone who works as a pseudo-journalist on a popular technology website would be able to tell the difference between an email service and a file storage service. You’d think he’d want to compare apples to apples and compare GMail’s 2.9 GB of Storage with Windows Live Hotmail’s 5 Gigabytes of Storage or compare the capabuilities of Microsoft’s SkyDrive with Google’s GDrive. 

Except Google hasn’t figured out how to ship GDrive for over 5 years so it would be an apples to vaporware comparison. Smile

Much love to my SkyDrive peeps on their new release. The champagne and ice cream yesterday was much appreciated. You can learn more about their release from the post Updates to Windows Live SkyDrive! on their team blog.  

Now playing: Foo Fighters - My Hero


 

October 4, 2007
@ 04:00 AM

Mini-Microsoft has a blog post up to let us know that his Facebook account was cancelled. In the comments he clarifies he wasn’t specifically targetted and this is just part of the Facebook terms of service. He writes

For those who probably will never see this Facebook help-topic, this is what I've been directed to:

http://www.facebook.com/help.php?page=45

The only relevant text that I can find:

"Facebook does not allow users to register with fake names, to impersonate any person or entity, or to falsely state or otherwise misrepresent themselves or their affiliations."

I imagine they only do something when someone complains vs. being constantly policing things. And someone out there (scanning the crowd of exceptionally good looking people who visit here) must have taken it upon themselves to complain.

I didn’t realize that if I don’t provide 100% accurate data about myself (thus making identity theft easier) I could get my account banned from Facebook.

I can understand why they want to encourage people to use real names since they want to be the kind of place that have users like “Dare Obasanjo” and “Robert Scoble” not ‘carnage4life’ and ‘scobleizer’ since the former implies a more personal experience.

However it seems dumb to be trying to replicate Friendster’s mistake by killing off every account that didn’t conform to their standards. There are ways to encourage such behavior without being jerks as they’ve clearly been in this case.

Now playing: Dem Franchize Boyz - Oh I Think They Like Me (remix) (feat. Jermaine Dupri, Da Brat & Lil Bow Wow)


 

Yesterday morning, I tossed out a hastily written post entitled It Must Be a Fun Time to Work on Microsoft Office which seems to have been misread by some folks based on some of the comments I’ve seen on my blog and in other places. So further exposition of some of the points in that post seems necessary.

First of all, there’s the question of who I was calling stupid when talking about the following announcements

  • Google announcing the launch of Presently, their Web-based Powerpoint clone. Interestingly enough, one would have expected presentation software to be the most obvious application to move to the Web first instead of the last.
  • Yahoo! announcing the purchase of Zimbra, a developer of a Web-based office productivity and collaboration suite.
  • Microsoft announcing the it would integrate Web-based storage and collaboration into it’s desktop office productivity suite.
  • IBM announcing that it would ship it’s own branded version of an Open Source clone of Microsoft’s desktop productivity suite.

Given that three of these announcements are about embracing the Web and the last one is about building disconnected desktop software, I assumed it was obvious who was jumping on a dying paradigm while the rest of the industry has already moved towards the next generation. To put this another way, James Robertson’s readers were right that I was talking about IBM.

There is something I did want to call out about James Robertson’s post. He wrote

People have moved on to the 80% solution that is the web UI, because the other advantages outweigh that loss of "richness".

I don’t believe that statement when it comes to office productivity software. I believe that the advantages of leveraging the Web are clear. From my perspective

  1. universal access to my data from any device or platform 
  2. enabling collaboration with “zero install” requirements on collaborators

are clear advantages that Web-based office productivity software has over disconnected desktop software.

It should be noted that neither of these advantages requires that the user interface is Web-based or that it is rich (i.e. AJAX or Flash if it is Web-based). Both of these things help but they aren’t a hard requirement.

What is important is universal access to my data via the Web. The reason I don’t have an iPhone is because I’m hooked on my Windows Mobile device because of the rich integration it has with my work email, calendar and tasks list. The applications on my phone aren’t Web-based, they are the equivalent of “desktop applications” for my phone. Secondly, I didn’t have to install them because they were already on my phone [actually I did have to install Oxios ToDo List but that’s only because the out-of-the-box task list synchronization in Windows Mobile 5 was less than optimal for my needs].

I used to think that having a Web-based interface was also inevitable but that position softened once I realized that you’ll need offline support which means building support for local storage + synchronization into the application (e.g. Google Reader's offline mode) to truly hit the 80/20 point for most people given how popular laptops are these days. However once you’ve built that platform, the same storage and synchronization engine could be used by a desktop application as well.

In that case, either way I get what I want. So desktop vs. Web-based UI doesn’t matter since they both have to stretch themselves to meet my needs. But it is probably a shorter jump to Web-enable the desktop applications than it is to offline-enable the Web applications.  

Now playing: Playa Fly - Feel Me


 

This is one of those posts I started before I went on my honeymoon and never got around to finishing. There are lots of interesting things happening in the world of office productivity software these days. Here are four announcements from the past three weeks that show just how things are heating up in this space, especially if you agree with Steve Gillmor that Office is Dead *(see footnote).

From the article Google Expands Online Software Suite 

MOUNTAIN VIEW, Calif. (AP) — Google Inc. has expanded its online suite of office software to include a business presentation tool similar to Microsoft Corp.'s popular PowerPoint, adding the latest twist in a high-stakes rivalry.

Google's software suite already included word processing, spreadsheet and calendar management programs. Microsoft has been reaping huge profits from similar applications for years.

Unlike Google's applications, Microsoft's programs are usually installed directly on the hard drives of computers.

From the article I.B.M. to Offer Office Software Free in Challenge to Microsoft’s Line

I.B.M. plans to mount its most ambitious challenge in years to Microsoft’s dominance of personal computer software, by offering free programs for word processing, spreadsheets and presentations.

Steven A. Mills, senior vice president of I.B.M.’s software group, said the programs promote an open-source document format.

The company is announcing the desktop software, called I.B.M. Lotus Symphony, at an event today in New York. The programs will be available as free downloads from the I.B.M. Web site.

From the blog post Yahoo scoops up Zimbra for $350 million

Yahoo has been on an acquisition binge late, but mostly to expand its advertising business. Now Yahoo is buying its way deeper into the applications business with the acquisition of Zimbra for a reported $350 million, mostly in cash. Zimbra developed a leading edge, Web 2.0 open source messaging and collaboration software suite, with email, calendar, document processing and a spreadsheet.

and finally, from the press release Microsoft Charts Its Software Services Strategy and Road Map for Businesses

 Today Microsoft also unveiled the following:

  • Microsoft® Office Live Workspace, a new Web-based feature of Microsoft Office that lets people access their documents online and share their work with others

Office Live Workspace: New Web Functionality for Microsoft Office

Office Live Workspace is among the first entries in the new wave of online services. Available at no charge, Office Live Workspace lets people do the following:

  • Access documents anywhere. Users can organize documents and projects for work, school and home online, and work on them from almost any computer even one not connected to the company or school network. They can save more than 1,000 Microsoft Office documents to one place online and access them via the Web.
  • Share with others. Users can work collaboratively on a project with others in a password-protected, invitation-only online workspace, helping to eliminate version-control challenges when e-mailing drafts to multiple people. Collaborators who don’t have a desktop version of Microsoft Office software can still view and comment on the document in a browser.

As you can see one of these four announcements is not like the others. Since it isn’t fair to pick on the stupid, I’ll let you figure out which company is jumping on a dying paradigm while the rest of the industry has already moved towards the next generation.  The Web is no longer the future of computing, computing is now about the Web.

* I do. Disconnected desktop software needs to go the way of the dodo.

Now playing: Prince - Sign 'O' the Times


 

I scored an invite to FriendFeed and after trying out the service, I have to say it is both disappointing and encouraging at the same time. It is disappointing because one would expect folks like Bret Taylor and Paul Buchheit who helped launch Google Maps, Gmail and AdSense while at Google to come up with something more innovative than a knock-off of Plaxo Pulse and Google’s SocialStream which are themselves knock-offs of the Facebook News feed.

On the other hand, this is encouraging because it is another example of how the digital lifestyle aggregator is no longer just a far out idea being tossed around on Marc Canter’s blog but instead has become a legitimate product category.  

So what exactly is FriendFeed? The site enables users to associate themselves with the various user generated content (UGC) sites which they use regularly that publish RSS feeds or provide open APIs and then this is turned into the equivalent of a Facebook Mini Feed for the user. You can get a good idea of it by viewing my page at http://friendfeed.com/carnage4life which aggregates the recent activities from my profiles on reddit, digg, and youtube.

The “innovation” with FriendFeed is that instead of asking you to provide the URLs of your RSS feeds, the site figures out your RSS feed from your username on the target service. See the screenshot below for this in action

Of course, this same “innovation” exists in Plaxo Pulse so this isn’t mindblowing. If anything, FriendFeed is currently a less feature rich version of Plaxo Pulse.

I personally doubt that this site will catch on because it suffers from the same chicken and egg problem that face all social networking sites that depend on network effects. And if it does catch on, given that there is zero barrier to entry in the feature-set they provide, I wouldn’t be surprised to see Facebook and a host of other services roll this into their feature set. I expect that News Feed style pages will eventually show up in a majority of social sites, in much the same way that practically every website these days has a friend’s list and encourages user generated content. It’s just going to be another feature when it comes to making a website, kinda like using tabs for navigation.

I’m sure Marc Canter finds this validation of his vision quite amusing.

Now playing: Puddle of Mudd - Control


 

Last month there was a press release published by Sophos, an IT  security company, with the tantalzing title Sophos Facebook ID probe shows 41% of users happy to reveal all to potential identity thieves which reports the following

 The Sophos Facebook ID Probe involved creating a fabricated Facebook profile before sending out friend requests* to individuals chosen at random from across the globe.
...

Sophos Facebook ID Probe findings:

  • 87 of the 200 Facebook users contacted responded to Freddi, with 82 leaking personal information (41% of those approached)
  • 72% of respondents divulged one or more email address
  • 84% of respondents listed their full date of birth
  • 87% of respondents provided details about their education or workplace
  • 78% of respondents listed their current address or location
  • 23% of respondents listed their current phone number
  • 26% of respondents provided their instant messaging screenname

In the majority of cases, Freddi was able to gain access to respondents' photos of family and friends, information about likes/dislikes, hobbies, employer details and other personal facts. In addition, many users also disclosed the names of their spouses or partners, several included their complete résumés, while one user even divulged his mother's maiden name - information often requested by websites in order to retrieve account details.

This is another example of how Facebook needs to be better at managing multiple social contexts. Right now, there is no way for me to alter my privacy settings to prevent people who I’ve added to my “friends list” from seeing my personal information. The thing is my “friends list” comprised of more than just friends. It is comprised of co-workers, people who work at the same company, people I went to high school with, and close personal friends. There’s also the category of “people who read my blog or use RSS Bandit” that I generally tend to decline friend requests from. I don’t mind some of these people being able to access my personal information (e.g. cell phone number, email address, birthday, etc) but clearly I also don’t want every random person who reads my blog that wants to be my “friend” on Facebook to have access to this information. 

Is there a better way to do this? Below are screenshots of the permissions model we came up with for Profiles on MSN Spaces when I worked on the feature juxtaposed with the Profile permissions options on Facebook.

Facebook
Profile privacy settings on Facebook

 

 Windows Live Spaces
Profile privacy settings on Windows Live Spaces

Straightforward isn’t it? I suspect that the problem here is that the folks at Facebook are refusing to acknowledge that their user base is changing now that they’ve opened up. As danah boyd writes in her post SNS visibility norms (a response to Scoble) 

Facebook differentiated itself by being private, often irritatingly so. Hell, in the beginning Harvard kids couldn't interact with their friends at Yale, but that quickly changed. Teens and their parents worship Facebook for its privacy structures, often not realizing that joining the "Los Angeles" network is not exactly private. For college students and high school students, the school and location network are really meaningful and totally viable structural boundaries for sociability. Yet, the 25+ crowd doesn't really live in the same network boundaries. I'm constantly shifting between LA and SF as my city network. When I interview teens, 80%+ of their FB network is from their high school. Only 8% of my network is from Berkeley and the largest network (San Francisco) only comprises 17% of my network. Networks don't work for highly-mobile 25+ crowd because they don't live in pre-defined networks. (For once, I'm an example!)
...
I don't really understand why Facebook decided to make public search opt-out. OK, I do get it, but I don't like it. Those who want to be PUBLIC are more likely to change settings than those who chose Facebook for its perceived privacy. Why did Facebook go from default-to-privacy-protection to default-to-exposure? I guess I know the answer to this... it's all about philosophy.

The first excerpt illustrates the point well. Facebook worked well as a social tool in the rigid social contexts of high school and college but completely breaks down when you’re all grown up.  Of course, the Facebook folks know this is an issue for some of their users. However it may be a “problem” that they consider to be By Design and not a bug.

The second excerpt is there because I’m surprised that danah is unsure about why Facebook profiles will now appear in search results. There are a lot of people for whom their social network profile is their primary or only online presence. Even for me, besides my blog(s), my Facebook profile is the only online identity Web which I keep updated regularly. It totally makes sense for Facebook to capitalize on this by making it so that everytime you search for a person whose primary presence is on their site, you get an ad to join their service [since only the fact that the person has a Facebook profile is exposed]. In addition, if you want to contact the person directly, you’re a lot better off joining Facebook and sending the person a private message than posting a comment on their blog [if they have one] or hoping that they’ve exposed their email address somewhere on the Web that isn’t their profile.

Update: The ability to expose a Limited Profile does render moot a lot of the points I just raised above. However making it a separate option from the privacy settings for the profile and incorrectly stating that your friends can always see your contact information makes it less likely to be used by users who are concerned about their privacy. Another example of a design flaw that is likely considered to be By Design according to the Facebook team.

Now playing: Metallica - The Unforgiven


 

Yesterday the Wall Street Journal had an article entitled Why So Many Want to Create Facebook Applications which gives an overview of the burst of activity surrounding the three month old Facebook platform. If a gold rush, complete with dedicated VC funds targetting widget developers, around building embedded applications in a social networking site sounds weirdly familar to you, that’s because it is. This time last year people were saying the same thing about building MySpace widgets. The conventional wisdom at the time was that sites like YouTube (acquired for $1.65 billion) and PhotoBucket (acquired for $250 million) rose in popularity due to their MySpace widget strategy.

So, why would developers who’ve witnessed the success of companies developing MySpace widgets rush to target a competing social networking site that has less users and requires more code to integrate with the site? The answer is that MySpace made the mistake of thinking that they were a distribution channel instead of a platform. If you are a distribution channel, you hold all the cards. Without you, they have no customers. On the other hand, if you are a platform vendor you realize that it is a symbiotic relationship and you have to make people building on your platform successful because of [not in spite of] your efforts.

Here are the three classic mistakes the folks at MySpace made which made it possible for Facebook to steal their thunder and their widget developers.

  1. Actively Resent the Success of Developers on Your Platform: If you are a platform vendor, you want developers building on your platform to be successful. In contrast, MySpace’s executives publicly griped about the success of sites like YouTube and PhotoBucket that were “driven off the back of MySpace” and bragged about building competing services which would become as popular as them since “60%-70% of their traffice came from MySpace”. In a sign that things may have gotten out of hand is when MySpace blocked PhotoBucket widgets only to acquire the site a month later, indicating that this was an aggresive negotiation tactic intended to scare off potential buyers.  

  2. Limit the Revenue Opportunities of Developers on Your Platform: MySpace created all sorts of restrictions to make it difficult for widget developers to actually make money directly from the site. For one, they blocked any widget that contained advertising even though advertising is the primary way to make money on the Web. Secondly, they restricted the options widgets had in linking back to the widget developers website thus driving users to where they could actually show them ads. Instead of trying to create a win<->win situation for widget developers (MySpace gets free features thus more engagement from their users, widget developers get ad revenue and traffic) the company tipped the balance excessively in their favor with little upside for widget developers.  

  3. Do Not Invest in Your Platform: For a company that depends so much on developers building tiny applications that integrate into their site, it’s quite amazing that MySpace does not provide any APIs at all. Nor do they provide a structured way for their users to find, locate and install widgets. It turns out that Fox Interactive Media (MySpace’s parent company) did build a widget platform and gallery but due to internal politics these services are not integrated. In fact, one could say that MySpace has done as little as possible to making developing widgets for their platform a pleasant experience for developers or their users. 

This is pretty much the story of all successful technology platforms that fall out of favor. If you do not invest in your platform, it will become obsolete. If people are always scared that you will cut off their air supply out of jealousy, they’ll bolt the first chance they get. And if people can’t make money building on your platform, then there is no reason for them to be there in the first place. Don’t make the same mistakes.

Now playing: 50 Cent - Many Men (Wish Death)


 

The Facebook developer blog has a post entitled Change is Coming which details some of the changes they've made to the platform to handle malicious applications including

Requests

We will be deprecating the notifications.sendRequest API method. In its place, we will provide a standard invitation tool that allows users to select which friends they would like to send a request to. We are working hard on multiple versions of this tool to fit into different contexts. The tool will not have a "select all" button, but we hope it enables us to increase the maximum number of requests that can be sent out by a user. The standardized UI will hopefully make it easier for users to understand exactly what they are doing, and will save you the trouble of building it yourself.

Notifications

Soon we will be removing email functionality from notifications.send, though the API function itself will remain active. In the future, we may provide another way to contact users who have added your app, as we know that is important. Deceptive and misleading notifications will continue to be a focus for us, and we will continue to block applications which behave badly and we will continue to iterate on our automated spam detection tools. You will also see us working on ways to automatically block deceptive notifications.

It looks like some but not all of the most egregious behavior is being targetted which is good. Specifically, I  wonder what is meant by deprecating the notifications.sendRequest API. When I think of API deprecation, I think of @deprecated in Java and Obsolete in C#, neither of which prevent the API from being used.

One of my biggest gripes with the site is the number of “friend requests” I get from applications with no way to opt out of getting these requests. However it doesn’t seem that this has been eliminated. Instead an API is being replaced with a UI component but the API isn’t even going away. I hope there is a follow up post where they describe the opt-out options they’ve added to the site so users can opt-out of getting so many unsolicited requests.

Now playing: Big Pun - Punish Me


 

Robert Scoble has a blog post up entitled Why Mahalo, TechMeme, and Facebook are going to kick Google’s butt in four years where he argues that search based on social graphs (e.g. your Facebook relationships) or generated by humans (e.g. Mahalo) will eventually trump Google's algorithms. I'm not sure I'd predict the demise of Google but I do agree that the social graph can be used to improve search and other aspects of the Internet experience, in fact I agree so much that was the topic of my second ThinkWeek paper which I submitted earlier this year (Microsoft folks can find it here).

However I don’t think Google’s main threat from sites like Facebook is that they may one day build social graph powered search that beats Google’s algorithms. Instead it is that these sites are in direct conflict with Google’s mission to

organize the world's information and make it universally accessible and useful.

because they create lots of valuable content that Google can not access. Google has branched out of Web search into desktop search, enterprise search, Web-based email and enterprise application hosting all to fulfill this mission.

The problem that Google faces with Facebook is pointed out quite well in Jason Kottke’s post Facebook vs. AOL, redux where he writes

Think of it this way. Facebook is an intranet for you and your friends that just happens to be accessible without a VPN. If you're not a Facebook user, you can't do anything with the site...nearly everything published by their users is private. Google doesn't index any user-created information on Facebook.2

and in Jeff Atwood's post Avoiding Walled Gardens on the Internet which contains the following excerpt

I occasionally get requests to join private social networking sites, like LinkedIn or Facebook. I always politely decline…public services on the web, such as blogs, twitter, flickr, and so forth, are what we should invest our time in. And because it's public, we can leverage the immense power of internet search to tie it all-- and each other-- together.

What Jason and Jeff are inadvertantly pointing out is that once you join Facebook, you immediately start getting less value out of Google’s search engine. This is a problem that Google cannot let continue indefinitely if they plan to stay relevant as the Web’s #1 search engine.

What is also interesting is that thanks to efforts of Google employees like Mark Lucovsky, I can use Google search from within Facebook but without divine intervention I can’t get Facebook content from Google’s search engine. If I was an exec at Google, I’d worry a lot more about the growing trend of users creating Web content where it cannot be accessed by Google than all the “me too” efforts coming out of competitors like Microsoft and Yahoo!.

The way you get disrupted is by focusing on competitors who are just like you instead of actually watching the marketplace. I wonder how Google will react when they eventually realize how deep this problem runs?

Now playing: Metallica - Welcome Home (Sanitarium)


 

This morning there were a number of news stories about collaboration between Windows Live and Bebo. These news stories didn’t tell the whole story. Articles such as C|Net’s Bebo's new instant messaging is Microsoft-flavored and TechCrunch’s Windows Live Messaging Comes to Bebo give the impression that the announcement was about instant messaging. However there was much more to the announcement. The agreement between Windows Live and Bebo spans two areas; social network portability and interop between Web-based IM and Windows Live Messenger

  1. Social Network Portability: As I’ve mentioned before a common practice among social networking sites is to ask users for their log-in credentials for their email accounts so that the social networking sites can screen scrape the HTML for the address book and import the user’s contact list into the social networking site. There are a number of problems with this approach, the main one being that the user is simply moving data from one silo to another without being able to get their contact list back from the social network and into their email client. There’s also the problem that this approach makes users more susceptible to phishing since it encourages them to enter their log-in credentials on random sites.  Finally, the user isn’t in control of how much data is pulled from their address book by the social network or how often it is pulled.

    The agreement between Windows Live and Bebo enables users to utilize a single contact list across both sites. Their friends in Bebo will be available as their contacts in Windows Live and vice versa. This integration will be facilitated by the Windows Live Contacts API which implements a user-centric access control model where the user grants applications permission to access and otherwise manipulate their contact list.

  2. Web-based IM and Windows Live Messenger interoperability: Users of Bebo that are also Windows Live Messenger users can opt in to getting notifications from Bebo as alerts in their desktop IM client. In addition, these users can add an “IM Me” button to their profile which allows people browsing their profile on the Web to initiate an IM conversation with them using a Microsoft-provided Web IM widget on the Bebo website which communicates with the Windows Live Messenger client on the profile owner’s desktop.

    The above scenarios were demoed at this year's MIX '07 conference during the session Broaden Your Market with Windows Live. The current plan is for the APIs for interacting with the Windows Live Messenger service and the IM widgets that can be embedded within a non-Microsoft website that power this scenario to be available via http://dev.live.com in the near future.

At the end of the day, it is all about putting users in control. We don’t believe that a user’s social graph should be trapped in a roach motel of our creation. Instead users should be able to export their contact lists from our service on their own terms and should be able to grow their social graph within Windows Live without having to exclusively use our services.

It’s your data, not ours. If you want it, you can have it. Hopefully, the rest of the industry comes around to this sort of thinking sooner rather than later.

Stay tuned, there’s more to come.

Now playing: Gucci Mane - So Icy (feat. Young Jeezy)


 

Brad Fitzpatrick, the founder of LiveJournal, who recently left Six Apart for Google has published notes on what he's going to be working on moving forward. It is an interesting read entitled Brad's Thoughts on the Social Graph which contains the following excerpts

Currently if you're a new site that needs the social graph (e.g. dopplr.com) to provide one fun & useful feature (e.g. where are your friends traveling and when?), then you face a much bigger problem then just implementing your main feature. You also have to have usernames, passwords (or hopefully you use OpenID instead), a way to invite friends, add/remove friends, and the list goes on. So generally you have to ask for email addresses too, requiring you to send out address verification emails, etc. Then lost username/password emails. etc, etc. If I had to declare the problem statement succinctly, it'd be: People are getting sick of registering and re-declaring their friends on every site., but also: Developing "Social Applications" is too much work.

Facebook's answer seems to be that the world should just all be Facebook apps.
...
Goals:
1. Ultimately make the social graph a community asset, utilizing the data from all the different sites, but not depending on any company or organization as "the" central graph owner. 
  1. Establish a non-profit and open source software (with copyrights held by the non-profit) which collects, merges, and redistributes the graphs from all other social network sites into one global aggregated graph. This is then made available to other sites (or users) via both public APIs (for small/casual users) and downloadable data dumps, with an update stream / APIs, to get iterative updates to the graph (for larger users)
...
Non-Goals:
  1. The goal is not to replace Facebook. In fact, most people I've talked to love Facebook, just want a bit more of their already-public data to be more easily accessible, and want to mitigate site owners' fears about any single data/platform lock-in. Early talks with Facebook about participating in this project have been incredibly promising. 

It seems to me that Facebook is the new Microsoft in that there are now a significant amount of people who are either upset at the level of "lock-in" they have created or are just plain jealous of their "wealth" who have created dedicated efforts to break their hegemony. It'll be interesting watching this play out.

From my perspective, I'm skeptical of a lot of the talk about social network portability because the conversation rarely seems to be user centric. Usually it's creators of competing services who are angry about "lock-in" because they can't get a new user's contacts from another service and spam them to gain "viral growth" for their service. As for the various claims of social network overload only the power users and geeks who join a new social network service a month (WTF is Dopplr?) have this problem.

A real social network is a community and users don't change communities at the drop of a hat. What I find more interesting is being able to bridge these communities instead of worrying about the 1% of users who hop from community to community like crack addled humming birds skipping from flower to flower.

I'll put it this way, when it comes to email which is more important? The ability to send emails to people regardless of what email service or mail client they use or the ability to import your contact list from one free email service into another when you switch service providers?


 

Disclaimer: This blog post does not reflect future product announcements, technical strategy or advice from my employer. Disregard this disclaimer at your own risk.

In my previous post Some Thoughts on Open Social Networks, I gave my perspective on various definitions of "open social network" in response to the Wired article Slap in the Facebook: It's Time for Social Networks to Open Up. However there was one aspect of the article that I overlooked when I first read it. The first page of the article ends with the following exhortation.

We would like to place an open call to the web-programming community to solve this problem. We need a new framework based on open standards. Think of it as a structure that links individual sites and makes explicit social relationships, a way of defining micro social networks within the larger network of the web.

This is a problem that interests me personally. I have a Facebook profile while my fiancée has a MySpace profile. Since I’m now an active user of Facebook, I’d like her to be able to be part of my activities on the site such as being able to view my photos, read my wall posts and leave wall posts of her own. I could ask her to create a Facebook account, but I already asked her to create a profile on Windows Live Spaces so we could be friends on that service and quite frankly I don’t think she’ll find it reasonable if I keep asking her to jump from social network to social network because I happen to try out a lot of these services as part of my day job. So how can this problem be solved in the general case?

OpenID to the Rescue

This is exactly the kind of problem that OpenID was designed to solve.  The first thing to do is to make sure we all have the same general understanding of how OpenID works. It's basically the same model as Microsoft Passport Windows Live ID, Google Account Authentication for Web-Based Applications and Yahoo! Browser Based Authentication. A website redirects you to your identity provider, you authenticate yourself (i.e. login) on your identity providers site and then are redirected back to the referring site along with your authentication ticket. The ticket contains some information about you that can be used to uniquely identify you as well as some user data that may be of interest to the referring site (e.g. username).

So how does this help us? Let’s say MySpace was an OpenID provider which is a fancy way of saying that I can use my MySpace account to login to any site that accepts OpenIDs . And now let’s say Facebook was a site that accepted OpenIDs  as an identification scheme. This means that I could add my fiancée to the access control list of people who could view and interact with my profile on Facebook by using the URL to her MySpace profile as my identifier for her.  So when she tries to access my profile for the first time, she is directed to the Facebook login page where she has the option of logging in with her MySpace credentials. When she chooses this option she is directed to the MySpace login page. After logging into MySpace with proper credentials, she is redirected back to Facebook  and gets a pseudo-account on the service which allows her to participate in the site without having to go through an account creation process.

Now that the user has a pseudo-account on Facebook, wouldn’t it be nice if when someone clicked on them they got to see a Facebook profile? This is where OpenID Attribute Exchange can be put to use. You could define a set of required and optional attributes that are exchanged as part of social network interop using OpenID. So we can insert an extra step [which is may be hidden from the user] after the user is redirected to Facebook after logging into MySpace where the user’s profile information is requested. Here is an example of the kind of request that could be made by Facebook after a successful log-in attempt by a MySpace user.

openid.ns.ax=http://openid.net/srv/ax/1.0
openid.ax.type.fullname=http://example.com/openid/sn_schema/fullname
openid.ax.type.gender=http://example.com/openid/sn_schema/gender
openid.ax.type.relationship_status=http://example.com/openid/sn_schema/relationship_status
openid.ax.type.location=http://example.com/openid/sn_schema/location
openid.ax.type.looking_for=http://example.com/openid/sn_schema/looking_for
openid.ax.type.fav_music=http://example.com/openid/sn_schema/fav_music
openid.ax.count.fav_music=3
openid.ax.required=fullname,gender,location
openid.ax.if_available=relationship_status,looking_for,fav_music

which could return the following results

openid.ns.ax=http://openid.net/srv/ax/1.0
openid.ax.type.fullname=http://example.com/openid/sn_schema/fullname
openid.ax.type.gender=http://example.com/openid/sn_schema/gender
openid.ax.type.relationship_status=http://example.com/openid/sn_schema/relationship_status
openid.ax.type.location=http://example.com/openid/sn_schema/location
openid.ax.type.looking_for=http://example.com/openid/sn_schema/looking_for
openid.ax.type.fav_music=http://example.com/openid/sn_schema/fav_music
openid.ax.value.fullname=Jenna
openid.ax.value.gender=F
openid.ax.value.relationship_status=Single
openid.ax.value.location=Seattle, WA, United States
openid.ax.value.looking_for=Friends
openid.ax.value.fav_music=hiphop,country,pop
openid.ax.update_url=http://www.myspace.com/url_to_send_changes_made_to_profile

With the information returned by MySpace, one can now populate a place holder Facebook profile for the user.

Why This Will Never Happen

The question at the tip of your tongue is probably “If we can do this with OpenID today, how come I haven’t heard of anyone doing this yet?”.  As usual when it comes to interoperability, the primary reasons for lack of interoperability are business related and not technical.  When you look at the long list of Open ID providers, you may be notice that there is no similar long list of sites that accept OpenID  credentials. In fact, there is no such list of sites readily available because the number of them is an embarassing fraction of the number of sites that act as Open ID providers. Why this discrepancy?

If you look around, you’ll notice that the major online services such as Yahoo! via BBAuth, Microsoft via Passport Windows Live ID, and AOL via OpenID all provide ways for third party sites to accept user credentials from their sites. This increases the value of having an account on these services because it means now that I have a Microsoft Passport Windows Live ID I not only can log-in to various Microsoft properties across MSN and Windows Live but also non-Microsoft sites like Expedia. This increases the likelihood that I’ll get an account with the service which makes it more likely that I’ll be a regular user of the service which means $$$. On the other hand, accepting OpenIDs does the exact opposite. It actually reduces the incentive to create an account on the site which reduces the likelihood I’ll be a regular user of the site and less $$$. Why do you think there is no OpenID link on the AOL sign-in page even though the company is quick to brag about creating 63 million OpenIDs?

Why would Facebook implement a feature that reduced their user growth via network effects? Why would MySpace make it easy for sites to extract user profile information from their service? Because openness is great? Yeah…right.

Openness isn’t why Facebook is currently being valued at $6 billion nor is it why MySpace is currently expected to pull in about half a billion in revenue this year. These companies are doing just great being walled gardens and thanks to network effects, they will probably continue to do so unless something really disruptive happens.   

PS: Marc Canter asks if I can attend the Data Sharing Summit between Sept. 7th – 8th. I’m not sure I can since my wedding + honeymoon is next month. Consider this my contribution to the conversation if I don’t make it.

Now playing: Wu-Tang Clan - Can It Be All So Simple


 

A few weeks ago, one of our execs at work asked me to think about "open" social networks. Since my day job is working on the social networking platform that underlies Windows Live Spaces and other Windows Live properties, it makes sense that if anyone at Microsoft is thinking about making our social networks "open" it should be me. However I quickly hit a snag. After some quick reading around, I realized that there isn't really a common definition of what it means for a social networking service to be "open". Instead, it seems we have a collection of pet peeves that various aggrieved parties like to blame on lack of openness. For example, read the Wired article Slap in the Facebook: It's Time for Social Networks to Open Up and compare it to this post on Read/Write Web entitled PeopleAggregator and Open Social Network Systems. Both articles are about "open" social networks yet they focus on completely different things. Below are my opinions on the various definitions of "open" in the context of social networking sites

  1. Content Hosted on the Site Not Viewable By the General Public and not Indexed by Search Engines:  As a user of Facebook, I consider this a feature not a bug. I've mentioned in previous blog postings that I don't think it is a great idea that all the stuff being published by teenagers and college students on the Web today will be held against them for the rest of their lives. Especially since using search engines to do quick background searches on potential hires and dates is now commonplace. Personally, I've had several negative experiences posting personal content to the public Web including

    1. fresh of out of college, I posted a blog post about almost hooking up with some girl at a nightclub and a heated email discussion I had with someone at work. It was extremely awkward to have both topics come up in conversations with fellow coworkers over the next few days because they'd read my blog.
    2. a few months ago I posted some pictures from a recent trip to Nigeria and this ignited a firestorm of over a hundred angry comments filled with abuse and threats to myself and my family because some Nigerians were upset that the president of Nigeria has servants domestic staff. I eventually made the pictures non-public on Flickr after conferring with my family members in Nigeria.
    3. around the same time I posted some pictures of my fiancée and I on my Windows Live Space and each picture now has a derogatory comment attached to it.

    At this point I've given up on posting personal pictures or diary like postings on the public Web. Facebook is now where I share pictures.

    When we first launched Windows Live Spaces, there was a lot of concern across the division when people realized that a significant portion of our user base was teenage girls who used the site to post personal details about themselves including pictures of themselves and friends. At the end we decided, like Facebook, that the default accessibility for content created by our teenage users (i.e. if they declare their age in their profile) would be for it to only be visible to people in their social network (i.e. Windows Live Messenger buddies and people in their Windows Live Spaces friends list). I think it is actually pretty slick that on Facebook, you can also create access control lists with entries like "anyone who's proved they work at Microsoft". 

  2. Inability to Export My Content from the Social Network: This is something that geeks complain about especially since they tend to join new social networking sites on a new basis but for the most part there isn't a lot of end user demand for this kind of functionality based on my experience working closely with the folks behind Windows Live Spaces and keeping an eye on feedback about other social networking sites. There are two main reasons for this, the first is that there is little value of having the content that is unique to the social network site outside of the service. For example, my friends list on Facebook is only useful in the context of that site. The only use for it outside the service would be for a way to bootstrap a new friends list by spamming all my friends on Facebook to tell them to join the new site.  Secondly, danah boyd has pointed out in her research that many young users of social networking sites consider their profiles to be ephemeral, to them not being able to just port your profile from MySpace to Facebook isn't a big deal because you're starting over anyway. For working professionals, things are a little different since they may have created content that has value outside the service (e.g. work-related blog postings related to their field of endeavor) so allowing data export in that context actually does serve a legitimate user need. 
  3. Full APIs for Extracting and Creating Content on the Social Network: With the growth in popularity and valuations of social networking sites, some companies have come to the conclusion that the there is an opportunity for making money by becoming meta-social network sites which aggregate a user's profiles and content from multiple social networking sites. There are literally dozens of Social Network Profile aggregators today and it is hard to imagine social networking sites viewing them as anything other than leeches trying to steal their page views by treating them as dumb storage systems. This is another reason why most social network services primarily focus on building widget platforms or APIs that enable you to create content or applications hosted within the site but don't give many ways to programmatically get content out.  

    Counter examples to this kind of thinking are Flickr and YouTube which both provide lots of ways to get content in and out of their service yet became two of the fastest growing and most admired websites in their respective categories. It is clear that a well-thought out API strategy that drives people to your site while not restricting your users combined with a great user experience on your website is a winning combination. Unfortunately, it's easier said than done.

  4. Being able to Interact with People from Different Social Networks from Your Preferred Social Network: I'm on Facebook and my fiancée is on MySpace. Wouldn't it be great if we could friend each other and send private messages without both being on the same service?

    It is likely that there is a lot of unvoiced demand for this functionality but it likely won't happen anytime soon for business reasons not technical ones. I suspect that the concept of "social network interop" will eventually mirror the current situation in the instant messaging world today.

    • We'll have two or three dominant social networking services with varying popularity in different global markets with a few local markets being dominated by local products.
    • There'll be little incentive for a dominant player to want to interoperate with smaller players. If interop happens it will be between players that are roughly the same size or have around the same market strength.
    • A small percentage of power users will use services that aggregate their profiles across social networks to get the benefits of social network interoperability. The dominant social networking sites will likely ignore these services unless they start getting too popular.
    • Corporate customers may be able to cut special deals so that their usage of public social networking services does interoperate with  whatever technology they use internally.

    Since I've assumed that some level of interoperability across social networking sites is inevitable, the question then is what is this functionality and what would the API/protocols look like? Good question.


 

August 2, 2007
@ 02:40 AM

Yesterday, I was chatting with a former co-worker about Mary Jo Foley's article Could a startup beat Microsoft and Google to market with a ‘cloud OS’? and I pointed out that it was hard to make sense of the story because she seemed to be conflating multiple concepts then calling all of them a "cloud OS". It seems she isn’t the only one who throws around muddy definitions of this term as evidenced by C|Net articles like Cloud OS still pie in the sky and blog posts from Microsoft employees like Windows Cloud! What Would It Look Like!? 

I have no idea what Microsoft is actually working on in this space and even if I did I couldn't talk about it anyway. However I do think it is a good idea for people to have a clear idea of what they are talking about when the throw around terms like "cloud OS" or "cloud platform" so we don't end up with another useless term like SOA which means a different thing to each person who talks about it. Below are the three main ideas people often identify as a "Web OS", "cloud OS" or "cloud platform" and examples of companies executing on that vision.

WIMP Desktop Environment Implemented as a Rich Internet Application (The YouOS Strategy)

Porting the windows, icons, menu and pointer (WIMP) user interface which has defined desktop computing for the last three decades to the Web is seen by many logical extension of the desktop operating system. This is a throwback to the Oracle's network computer of the late 1990s where the expectation is that the average PC is not much more than a dumb terminal with enough horse power to handle the display requirements and computational needs of whatever rich internet application platform is needed to make this work.

A great example of a product in this space is YouOS. This seems to be the definition idea of a "cloud os" that is used by Ina Fried in the C|Net article Cloud OS still pie in the sky.

My thoughts on YouOS and applications like it were posted a year ago, my opinion hasn't changed since then.

Platform for Building Web-based Applications (The Amazon Strategy)

When you look at presentations on scaling popular websites like YouTube, Twitter, Flickr, eBay, etc it seems everyone keeps hitting the same problems and reinventing the same wheels. They all start of using LAMP thinking that’s the main platform decision they have to make. Then they eventually add on memcached or something similar to reduce disk I/O. After that, they may start to hit the limits of the capability of relational database management systems and may start taking data out of their databases, denormalizing them or simply repartition/reshard them as they add new machines or clusters. Then they realize that they now have dozens of machines in their data center when they started with one or two and managing them (i.e. patches, upgrades, hard disk crashes, dynamically adding new machines to the cluster, etc) becomes a problem.

Now what if someone who’d already built a massively scalable website and now had amassed a bunch of technologies and expertise at solving these problems decided to rent out access to their platform to startups and businesses who didn’t want to deal with a lot of the costs and complexities of building a popular Website beyond deciding whether to go with LAMP or WISC? That’s what Amazon has done with Amazon Web Services such as EC2 ,S3, SQS and the upcoming Dynamo.  

The same way a desktop operating system provides an abstraction over the complexity of interacting directly with the hardware is the way Amazon’s “cloud operating system” insulates Web developers from a lot of the concerns that currently plague Web development outside of actually writing the application code and dealing with support calls from their customers.

My thoughts on Amazon’s Web Services strategy remain the same. I think this is the future of Web platforms but there is still a long way to go for it to be attractive to today’s startup or business.

NOTE: Some people have commented that it is weird for an online retailer to get into this business. This belies a lack of knowledge of the company’s history. Amazon has always been about gaining expertise at some part of the Web retailer value chain then opening that up to others as a platform. Previous examples include the Amazon Honor System which treats their payment system as a platforn, Fulfillment by Amazon which treats their warehousing and product shipping system as a platform, zShops allows you to sell your products on their Website as well as more traditional co-branding deals where other sites reused their e-commerce platform such as Borders.com.

Web-based Applications and APIs for Integrating with Them (The Google Strategy)

Similar to Amazon, Google has created a rich set of tools and expertise at building and managing large scale websites. Unlike Amazon, Google has not indicated an interest in renting out these technologies and expertise to startups and businesses. Instead Google has focused on using their platform to give them a competitive advantage in the time to market, scalability and capabilities of their end user applications. Consider the following… 

If I use GMail for e-mail, Google Docs & Spreadsheets for my business documents, Google Calendar for my schedule, Google Talk for talking to my friends, Google search to find things on my desktop or on the Web and iGoogle as my start page when I get on the computer then it could be argued that for all intents and purposes my primary operating system was Google not Windows. Since every useful application eventually becomes a platform, Google’s Web-based applications are no exception. There is now a massive list of APIs for interacting and integrating with Google’s applications which make it easier to get data into Google’s services (e.g. the various GData APIs) or to spread the reach of Google’s services to sites they don’t control  (e.g. widgets like the Google AJAX Search API and the Google Maps API).

In his blog post GooOS, the Google Operating System Jason Kottke argues that the combination of Google’s various Web applications and APIs [especially if they include an office suite] plus some desktop and mobile entry points into their services is effectively a Google operating system. Considering Google’s recent Web office leanings and its bundling deals with Dell and Apple, it seems Jason Kottke was particularly prescient given that he wrote his blog post in 2004.

Now playing: Fabolous - Do The Damn Thing (feat. Young Jeezy)


 

Jerff Atwood has a blog post entitled Google's Number One UI Mistake where he writes

Google's user interface minimalism is admirable. But there's one part of their homepage UI, downloaded millions of times per day, that leaves me scratching my head:

Google: I'm not feeling so lucky.

Does anyone actually use the "I'm Feeling Lucky" button? I've been an avid Google user since 2000; I use it somewhere between dozens and hundreds of times per day. But I can count on one hand the number of times I've clicked on the "I'm Feeling Lucky" button.
...
I urge us to Omit Needless Buttons. I hope the "I'm Feeling Lucky" button isn't considered
a sacred cow at Google. Removing it would be one small step for Google, but a giant collective improvement in the default search user interface for users around the world.

A number of Jeff’s readers chimed in with the reason why Jeff is mistaken about this being a user interface “mistake” on Google’s part.

Robert Cooper wrote the following as part of his response

Google actually brought that up when they were user testing the UI a while back. (I can't remember where I read the story) But the crux of it was, the users actually LIKE having that button there, and rated the search interface lower when it wasn't there.

Geoff Wilson also validates the idea that this is on purpose in his response which is excerpted below

I remember listening to a google staffer on a podcast a while ago, and the reason the "I feel lucky" button stays is not its click through rates, but the message it sends about google's corporate culture.

Apparently, users felt Google was more human by having something quirky like that on the front page. It doesn't have lots of people click on it, but it encouraged more people to click on the main "Google Search" button.

and here’s one more comment excerpt, this time from Max

They can't remove that button now, it's part of their branding, part of their identity. They would be no more likely to stop updating their logo with colorful playful holiday-appropriate images (the shock of all that wasted bandwidth as proxy servers everywhere have to recache it!) than they would to remove that button.

Needless to say, the connection that branding builds between the user and the service/company is important. This button and the feeling of playfulness that it reflects on google even moreso. The few extra bytes of page size and the small extra cognitive load is more than made up for the positive feelings it engenders in users.

 Building a connection with the users of your software is important. There’s nothing like a little playfulness and humor to make your company and your software seem a lot more friendly to your customers.

Now playing: Timbaland - The Way I Are (Extended mix) (feat. Keri Hilson, Sebastian & D.O.E)


 

Yesterday I got offered an opportunity to interview Vint Cerf just before he gave his talk entitled Tracking the Internet into the 21st Century (link is to video of the talk) at Google's Kirkland offices. I got to ask the questions I blogged about yesterday and also learned about some of Vint Cerf's interests in Nigeria. Below are the questions I asked and his paraphrased answers to my questions.

Yesterday I got offered an opportunity to interview Vint Cerf just before he gave his talk entitled Tracking the Internet into the 21st Century (link is to video of the talk) at Google's Kirkland offices. I got to ask the questions I blogged about yesterday and also learned about some of Vint Cerf's interests in Nigeria. Below are the questions I asked and his paraphrased answers to my questions.

Q: Why did he decide to leave MCI, a company steeped in networking technology, to join Google, an advertising and search engine company, as Chief Internet Evangelist

A: The job title was not his doing. Larry, Sergey and Eric told him they wanted him to continue his efforts in encouraging the growth of the Internet around the world and thought the title "Chief Internet Evangelist" best fit this position. There are 6.5 billion people on the planet today yet there are only 1 billion people on the Internet. Google would like to see the other 5 billion people on the Internet because the more people there are using the Internet, the better it is for Google. This is why the company needed a "Chief Internet Evangelist".

Vint Cerf spends a significant portion of his time encouraging implementations of the Internet. He travels all over the world meeting with senior government officials (presidents, ministers of information, etc) to recommend Internet friendly policies, discourage the rise of monopolistic or closed networks and encourage domestic/foreign investments in fledgling markets where Internet usage hasn't taken off. For example, he is working with some charitable entities to donate solar powered Internet cafes to businesses in Nigeria to encourage the usage of the Internet in remote or underprivileged parts of the country.

One aspect of the Internet's growth which he didn't pay much attention to at first but does now is Internet enabled mobile phones. It is estimated that there will be 3 billion people with mobile phones by the end of the year. That is 3 billion people who could all be connected to the Internet, if Internet connectivity became ubiqitous on mobile devices within a few years.

Looking back at the past few years, it is clear that adding more users to the Internet increases the quantity and diversity of information on the Web. This trend has been hastened by the rise of the consumer as producer. We now have people who would be traditionally considered to be consumers producing content on blogs, video sharing sites like YouTube, and creating social networks on sites like Orkut and Facebook. Another interesting trend is the rise of virtual worlds like World of Warcraft and Second Life. In these worlds users are creating interesting economic and sociological experiments with fascinating consequences (e.g. gold farming in China). In fact, some college professors are encouraging their students to join these sites to test out economic and sociological theories in ways that simply weren't feasible in the past. An interesting idea would be to see if we could create virtual objects which were associated with and could influence objects in the real world. For example, a virtual university where the electron microscopes and telescope actually displayed image data from electron microscopes and telescopes in the real world. Maybe as an optimization we could cache large amounts of the astronomical data so multiple instances of the virtual telescope could be used at once but only rarely would the physical telescope have to be used so there wasn't resource contention. Given that Google already has already started partnering with NASA to store and process large amounts of astronomical data this may be something that the company could be interested in trying out in the future.

Q: He has spoken out on Google's behalf in favor of net neutrality. However there seem to be many different definitions of Net Neutrality, some of which imply that having different tiers for Quality of Service is OK and some of which don't, which definition is Google in favor of and why?

A: Google didn't start the network neutrality debate, AT&T's CEO Ed Whitacre did when he claimed that companies like Google are getting a "free ride" on his network. This seems backwards to Vint Cerf since AT&T's customers pay broadband fees so they can access any site on the Internet. Expecting companies to pay AT&T's for access to its paying customers who are already paying for access to the Internet is old school "Telephone Think" that harkens back to the monopoly days of Ma Bell.

The philosophy of the Internet comes from a completely different roots. The philosophy was pretty much "Here are the specs, if you can figure out how to implement our protocols and can connect to our network then it's all good". This open philosophy is what enabled the growth of the Internet and eventually led to commercial entities [including telcos like AT&T] to become part of the network.

Vint Cerf and Google's definition of network neutrality has these five basic pillars

  1. users should be able to reach any service connected to the network
  2. users should be able to run any application and connect to the network (of course, this doesn't apply to applications that violate the law)
  3. it is OK to charge for higher speed connections to the network.
  4. operators should not discriminate against services a user is trying to access by varying the user's QoS or access charges when accessing that service.
  5. Discrimination against a type of service (e.g. all video traffic has different QoS) is OK but singling out specific sites is not.

A number of ISPs already break these rules yet are not upfront with users that they are not getting a full Internet experience. Some claim that these rules limit the ability of ISPs to prevent denial of service attacks, fight spam and perform other activities that protect their networks. Google believes that such protections can still be enforced but should be done at the application layer and not by discriminating against packets. As for ISPs that believe this limits their ability to provide value added services [such as video sharing] the response is that competition should be based on providing innovative services instead of by artificially limiting the capabilities of your competitors because you control the network pipes.

Google wants the Internet to be an open environment which allows for innovation. They believe this is important to the Internet's growth.

Q: Google just pledged to spending up to $4.6 billion to license the 700MHz wireless spectrum in what the company has described as the most significant auction of wireless spectrum in history by the U.S. federal government. Why is this auction so significant and what kind of services can we expect from Google if it wins the auction?

A: [Editor's Note: Why this auction is significant is summarized quite well in David Stone's post Vint Cerf and the 700MHz Spectrum]
Google's primary goal is to increase the openness of Internet-connected networks around the world. This is why they've committed at least $4.6 billion to licensing the 700MHz wireless spectrum.

It isn't quite clear what business model Google will use the 700MHz spectrum for if they win the auction. Whatever they end up deciding, it will honor the four principles of open platforms they have espoused with regards to wireless networks. It is quite likely that leasing out this capacity is one of the business models Google will try out. However due to the propagation characteristics of the 700MHz band, it is likely that different business models will have to apply in rural versus urban environments.

Q: Net neutrality gets a lot of press, however there are other issues facing the Internet as well. What keeps him up at night besides net neutrality? Botnets? Government censorship of the Internet? Concerns that we’ll never upgrade from the current version of the Internet since it is already so entrenched around the world?

A: The rise of botnets, domain name security and the problems related to handling internationalized domain names (IDNs) are at the top of the list of problems facing the Internet that concern Vint Cerf. The IDN problem is particularly pernicious because not only did we have to figure out how to support non-ASCII characters in a system that was never designed to support them but once a way was found, the IDN homograph attack was born which promptly reverse most of the gains.

Switching to IPv6 is also an issue facing the Internet that we will have to deal with sooner than most people expect. Some have predicted that at the current rate of allocation by ICAAN we will run out of IPv4 addresses by 2011. At that point, it will start to look a lot more attractive to switch to IPv6. There may be workarounds such as people leasing some of the blocks they've been allocated to other parties but this leads to interesting problems for routers since the routing tables will be screwed up and will have to be tampered with to adjust to these shenanigans. Given that pretty much all the major operating systems (Vista, Mac OS X, *nix, etc) and networking equipment manufacturers (e.g. Juniper, Cisco) support IPv6, it's really up to the ISPs and they likely won't make any moves without customer demand. Unfortunately for them, things are liably to get ugly in the next five years or so and they may have to change their minds.


 

One of the side effects of working for a large, successful, multinational corporation is that you tend to lose your sense of perspective. For example, take this post from the Official Google blog entitled Cookies: expiring sooner to improve privacy which states

We are committed to an ongoing process to improve our privacy practices, and have recently taken a closer look at the question of cookie privacy. How long should a web site "remember" cookie information in its logs after a user's visit? And when should a cookie expire on your computer? Cookie privacy is both a server and a client issue.

On the server side, we recently announced that we will anonymize our search server logs — including IP addresses and cookie ID numbers — after 18 months.
...
In the coming months, Google will start issuing our users cookies that will be set to auto-expire after 2 years, while auto-renewing the cookies of active users during this time period. In other words, users who do not return to Google will have their cookies auto-expire after 2 years. Regular Google users will have their cookies auto-renew, so that their preferences are not lost. And, as always, all users will still be able to control their cookies at any time via their browsers.

What’s is interesting in this post is that Google has sidestepped the actual privacy issue that has many people concerned about the amount of knowledge the company has about Internet users. Numerous bloggers such as Nelson MinarShelley Powers and John Dowdell have already pointed how this change doesn't actually change the status quo. In today’s world, Google knows more about most Internet users than their spouse. Thanks to the magic of HTTP cookies Google knows remembers...

You pretty much can't use the Web without running into a Google cookie. So it seems somewhat facetious for Google to claim that if you can avoid using the Internet for two years then they'll forget everything they are storing about you. Oops, actually they don't even claim that. They simply claim that they’ll stop associating your old data with your current usage, if you manage to avoid hitting a Google cookie for two years. 

If Google really wanted to address people's privacy concerns they’d blog about how they plan to use and protect all the data they are collecting about Internet users from all of their services instead of making ineffective token gestures that are specific to one service.      

Now playing: Lil Boosie & Webbie - Wipe Me Down (feat. Foxx)


 

Yesterday I saw a blog post by John Battelle where he said he turned down a chance to be part of discussion on a television show about whether Facebook is the next Google. This seems like a pretty absurd assertion on the face of it and the reason is snuggled somewhere in the middle of Robert Scoble's post Why Facebook, why now? where he writes

Everything I do in Facebook is about interacting with people. For instance, at the top of my Facebook inbox right now is Ryan Coomer. The advertising next to him says “Try Forex Trading Today.” There is absolutely NO connection between who Ryan is and the advertising that’s put next to him.

Imagine if advertisers could “buy people.” I just clicked on Ryan’s profile, hes into Running and Golf. Why don’t ads for running and golf gear get put onto his profile? Wouldn’t that make sense? He’s also a software developer. Where’s the Visual Studio advertisement? He’s into video games. Where’s the Halo 3 advertisement?

Translation: Facebook needs an advertising platform and it needs one in the worst way. I’m not going to even look at the ads until the ads are tied to the people on Facebook. Facebook knows what we’re into, put ads for those things onto our profiles and messages.

Robert Scoble is both right and wrong. He is right that Facebook needs an advertising platform but he is wrong about how it should be used. If Facebook has detailed demographic data about all its users then it makes the most sense to show the user what they are most interested in and not what the person they are currently interacting with on the site is most interested in. That's counter productive. I hate running and golf, I like G-Unit and alcopops. Which ads does it make more sense to show me if I'm browsing Ryan Coomer's profile?

Anyway, that's besides the point. Let's go back and look at what made Google such a compelling investment when they had their IPO. The company had

  1. An extremely popular website that served a key need for its users. 
  2. A way to make a metric ton of money from their popularity via search ads.

#1 is obvious but #2 needs further clarification. Search advertising created a paradigm shift in the advertising market in several ways. For one, it was the first time in the history of advertising that advertisers had a way to conclusively calculate the ROI of their advertising campaigns compared to other media (TV, newspapers, radio, sponsorships, etc). More importantly, it allowed advertisers to target people when they were in the process of making a buying related decision (i.e. doing research). For the most part advertising is often intrusive, irrelevant and often just plain annoying. On the other hand, ads shown when you are doing research for a commercial transaction are exactly what you want. It is quite telling when you look at highest paying search terms and see that it is for terms like "mortgage refinance", "dui" and "consolidate loans". When searching for those terms I want to see ads, in fact the more ads the better.  What is most interesting is that even though the majority of searches on the Web are non-commercial, this was still a goldmine which could only get more lucrative as more people got on the Web and advertisers eventually realized that their dollars needed to follow the eyeballs and search advertising was actually an easier expense to justify than all their other ad campaigns.

Now we know what made Google such an attractive investment back in the day which has now turned itself into a major force in the software industry and on the Web. So what does Facebook have?

  1. An extremely popular website that served a key need for its users. 
  2. The hope that way will be able to make a metric ton of money from their popularity via highly targeted ads based on all the rich demographic data they have about their users.

The initial thrust of #2 is that Facebook hasn't figured out how to make money yet. More importantly, if they do figure out how to build an ad platform it will likely be a system that is more complex than Google's Adwords and AdSense combined. When you think about it Adwords is pretty simple, allow people to bid on search terms then show these people's ads when people use those search terms. AdSense is similarly straightforward, use term extraction to convert the content on the page to the equivalent of search terms and show ads from the people who bid on ads. Of course, the devil is in the details and is mainly about optimizing how much money can be extracted from their advertisers.

On the other hand, if you want to show display ads based on demographic data you'll also need to build something very similar to a recommendation system as well as everything else I've described because you want to know that if I like wearing G-Unit clothing that I probably wouldn't mind seeing ads for Ecko Unlimited wear either or that if I like music by Young Jeezy that I probably would like music from T.I. or that if I've read Freakonomics I probably wouldn't mind reading The Tipping Point.

Now let's assume that the developers at Facebook figure out how to build an advertising platform that is more complex than Google's Adwords and AdSense combined. There is still the question of whether the "spray and pray" approach that is the hallmark of display advertising [even if you are sure that the ads will be about the topics the user is interested in]  is a more lucrative model than showing people ads when they are actually looking for something via search ads. I personally don't think it will be but I'm no expert and I'm definitely no fortune teller.

I suspect that the folks at Facebook will eventually realize how hard a problem it is to monetize their users in a way that justifies the exorbitant valuation that has now been placed on them. When that time comes, I wouldn't be surprised if they find some sucker to buy them based on the promise that they present as opposed to the value that they actually bring. Almost like when Yahoo! paid $5 billion for Broadcast.com which gave us the Mark Cuban we all know and love but also ended up as one of the top five worst billion dollar acquisitions of all time in the Internet space.


 

In my previous post I mentioned the various problems with relying on incubation teams to bring innovation into a product or organization. The obvious follow up question is that if carving off some subset of your team to work on the "next big thing" while the rest of your employees work on the boring bread and butter product(s) that pay the bills doesn't work, how do you revitalize an organizations products and make them innovative?

My advice is to look at companies within your industry that are considered innovative and see what you can learn from them. One such company is Google which is widely considered to be the most innovative company on Earth by many in the software industry. A number of Google's competitors have several internal groups whose job is to "incubate ideas" and foster innovation yet it seems that Google is the company most associated with innovation in the online space. For example, Yahoo! has Brickhouse and Yahoo! Research while Microsoft has Microsoft Research, Live Labs, Search Labs, and Windows Live Core among others. 

Below are some of the ways technology companies can follow their example without having to resort to some of their more eccentric practices like free food prepared by gourmet chefs and on-site massages, dry cleaning and oil changes to motivate your employees.

  1. Everyone is Responsible for Innovation: There are several ways Google has created a culture where every technical employee feels that innovation is expected of them. First, there is the strong preference for people who have a track record of producing original ideas such as Ph.D's [who are required to produce original research which advances the state of the art as part of their thesis] and founders of Open Source projects (e.g. Spencer Kimball (GIMP), Aaron Boodman (Greasemonkey), and Guido Van Rossum (Python)). Secondly, employees are strongly encouraged but not required to spend 20% of their time on projects of their own design which are intended to benefit the company and/or its customers. Not only does this give employees an outlet for their creativity in a productive way, working on multiple projects at once gives developers a broader world view which makes it less likely that they will develop tunnel vision with regards to their primary project. Finally, Google has a single code base for all of their projects and developers are strongly encouraged to fix bugs or add features to any Google product they want even if they are not on the product team. This attitude encourages the cross pollination of ideas across the company and encourages members of the various product teams to keep an open mind about ideas from outside their particular box.

  2. Good Ideas Often Come from Outside your Box: A lot of people in the software industry often criticize Microsoft for its practice of innovation through acquisition and have compiled lists of Microsoft's innovations that were actually acquisitions but the fact is that the road to success lies in being able to spot good ideas whether they come from within your company or without. Google is no exception to this rule as the following table of acquisitions and the Google products they resulted in shows

    Acquired Company/ProductGoogle Product
    Applied Semantics Google AdSense
    Kaltix Google Personalized Search
    Keyhole Corp. Google Earth
    Where2 Google Maps
    ZipDash Google Ride Finder
    2Web Technologies Google Spreadsheets
    Upstartle Google Docs
    Urchin Software Corporation + Measure Map Google Analytics
    Zenter + Tonic Systems Unreleased Google Web-based Presentation application

    As you can see from the above list, a lot of Google's much lauded products were actually the products of acquisitions as opposed to the results of internal incubation. Being able to conquer the NIH mentality is important if one wants to ensure that the products that exhibit the best ideas are produced by your company because quite often they won't originate from your company.  

  3. Force Competition to Face the Innovator's Dilemma: One reason that a number of Google's products are considered innovative is that they challenge a number of pre-existing notions about software in certain categories. For example, when Gmail [a product of an engineer's 20% time spent on side projects] was first launched it was a shock to see a free Web-based email service qive users 1 gigabyte of free storage. A key reason that this was a shock was because most free Web-based email services gave users less than a hundredth of that amount of storage. This was because the business model for free email was primarily to give users a crappy user experience (2MB of storage, obnoxious advertising, etc) and then charge them for upgrading to a decent experience. Thus there was little incentive for the major players in the free email business to give free users lots of storage or a rich online experience because isn't how the business worked. Another example, that is likely to be a classic case study of the innovator's dilemma in the years to come is Google Docs & Spreadsheets vs. Microsoft Office. From the Wikipedia article on disruptive technology

    In low-end disruption, the disruptor is focused initially on serving the least profitable customer, who is happy with a good enough product. This type of customer is not willing to pay premium for enhancements in product functionality. Once the disruptor has gained foot hold in this customer segment, it seeks to improve its profit margin. To get higher profit margins, the disruptor needs to enter the segment where the customer is willing to pay a little more for higher quality. To ensure this quality in its product, the disruptor needs to innovate. The incumbent will not do much to retain its share in a not so profitable segment, and will move up-market and focus on its more attractive customers. After a number of such encounters, the incumbent is squeezed into smaller markets than it was previously serving. And then finally the disruptive technology meets the demands of the most profitable segment and drives the established company out of the market.

    As someone who now maintains several wedding lists in collaboration with his future spouse I can say without a doubt that universal access to our files from any computer without my fiancèe or I having to install or purchase any software is head and shoulders beyond the solution provided by traditional desktop productivity suites. In addition, it is quite clear that Google will move to address the gaps in time (see Google Gears) so we are likely on the cusp of a multi-billion dollar software category undergo upheaval in the next few years.

    The main lesson here is Change the Game. Do not play by the rules that favor your competitors.

The key thing for people wanting to learn from Google's practices isn't to follow each of Google's specific policies but instead to understand the philosophy behind their practices then apply those philosophies in your specific context.


 

A couple of years ago, I wrote a blog post entitled Social Software: Finding Beauty in Walled Gardens where I riffed on the benefits of being able to tell the software applications you use regularly "here are the people I know, these are the ones I trust, etc". At the time I assumed that it would be one of the big Web companies such as Google, Yahoo!, or Microsoft that would build the killer social software platform that was powered by this unified view of your social connections. I was wrong. Facebook has beaten everyone to doing it first. There are a lot of user scenarios on the Web that can be improved if the applications we were using know who our friends, family and co-workers were knew without us having to explicitly tell them. Below are a couple of online services where access to a user's social network has made Facebook better at performing certain Web tasks than the traditional market leaders. 

NOTE: This started off as three different blog posts in my writing queue but after reading ridiculous overhype like ex-Google employees publicly decamping from their former employer because 'Facebook is the Google of yesterday, the Microsoft of long ago' I decided to scale back my writing about the service and merge all my thoughts into a single post. 

Displacing Email for Personal Communication

Gervase Markham, an employee of the Mozilla Foundation, recently wrote in his blog post entitled The Proprietarisation of Email

However, I also think we need to be aware of current attempts to make email closed and proprietary.

What am I talking about, I hear you ask? No-one's resurrected the idea of a spam-free email walled garden recently. Companies who tout their own secure mail protocols come and go and no-one notes their passing. The volume of legitimate email sent continues to grow. What's the worry?

I'm talking about the messaging systems built into sites like Facebook and LinkedIn. On several occasions recently, friends have chosen to get back in touch with me via one of these rather than by email. Another friend recently finished a conversation with a third party by saying "Facebook me"; when I asked her why she didn't just use email, she said "Oh, Facebook is so much easier".

And she's right. There's no spam, no risk of viruses or phishing, and you have a ready-made address book that you don't have to maintain. You can even do common mass email types like "Everyone, come to this event" using a much richer interface. Or other people can see what you say if you "write on their wall". In that light, the facts that the compose interface sucks even more than normal webmail, and that you don't have export access to a store of your own messages, don't seem quite so important.

After I read this post, I reflected on my casual use of the user to user messaging feature on Facebook and realized that even though I've only used it a handful of times, I've used it to communicate with friends and family a lot more than I have used either of my personal email addresses in the past three months. In fact, there are a bunch of friends and family whose email addresses I don't know that I've only communicated with online through Facebook. That's pretty wild. The fact that I don't get spam or random messages from people I don't know is also a nice plus and something a lot of other social network sites could learn from.

So one consequence of Facebook being used heavily by people in my real-life social network is that it is now more likely to be my interface for communicating with people I know personally than email. I suspect that if they ever add an instant messaging component to the site, it could significantly change the demographics of the top instant messaging applications.

Changing the Nature of Software Discovery and Distribution

I wrote about this yesterday in my post Marc Andreessen: The GoDaddy 2.0 Business Model but I think the ramifications of this are significant enough that it bears repeating. The viral software distribution model is probably one of the biggest innovations in the Facebook platform. Whenever my friends add an application to their dashboard I get a message in my news feed informing with a link to try out the application. I've tried out a couple of applications this way and it seems like a very novel and viral way to distribute applications. For one thing, it definitely a better way to discover new Facebook applications than browsing the application directory. Secondly, it also means that the best software is found a lot more quickly. The iLike folks have a blog post entitled Holy cow... 6mm users and growing 300k/day! they show a graph that indicates that iLike on Facebook has grown faster in its first few weeks than a number of popular services that grew quite quickly in their day including Skype, Hotmail, Kazaa and ICQ. 6 million new users in less than a month? 300,000 new users a day? Wow.

Although there are a number of issues to work out before transferring this idea to other contexts, I believe that this is a very compelling to approach how new software is discovered and distributed.  I would love it if I my friends and family got a notification whenever I discovered a useful Firefox add-on or a great Sidebar gadget and vice versa. I wouldn't be surprised if this concept starts showing up in other places very soon.  

Facebook Marketplace: A Craigslist Killer

Recently my former apartment was put up for rent after I broke the lease as part of the process of moving into a house. I had assumed that it would be listed in the local paper and apartment finding sites like Apartments.com. However I was surprised to find out from the property manager that they only listed apartments on Craig's List because it wasn't worth it to list anywhere else anymore. It seemed that somewhere along the line, the critical mass of apartment hunters had moved to using Craig's List for finding apartments instead of the local paper.

Since then I've used Craig's List and I was very dissatisfied with the experience. Besides the prehistoric user interface, I had to kiss a lot of frogs before finding a prince. I called about a ten people based on their listing and could only reach about half of them. Of those one person said he'd call back and didn't, another said he'd deliver to my place and then switched off his phone after I called to ask why he was late (eventually never showed) while yet another promised to show up then called back to cancel because his wife didn't want him leaving the house on a weekend. I guess it should be unsurprising how untrustworthy and flaky a bunch of the people listing goods and services for sale on Craig's List are since it doesn't cost anything to create a listing.

Now imagine if I could get goods and services only from people I know, people they know or from a somewhat trusted circle (e.g. people who work for the same employer) wouldn't that lead to a better user experience than what I had to deal with on Craig's List? In fact, this was the motivation behind Microsoft's Windows Live Expo which is billed as a "social marketplace". However the problem with marketplaces is that you need a critical mass of buyers and sellers for them to thrive. Enter Facebook Marketplace

Of course, this isn't a slam dunk for Facebook and in fact right now there are ten times as many items listed for sale in the Seattle area on Windows Live Expo than on Facebook Marketplace (5210 vs. 520). Even more interesting is that a number of listings on Facebook Marketplace actually link back to listings on Craig's List which implies that people aren't taking it seriously as a listing service yet.

Conclusion

From the above, it is clear that there is a lot of opportunity for Facebook to dominate and change a number of online markets beyond just social networking sites. However it is not a done deal. The company is estimated to have about 200 employees and that isn't a lot in the grand scheme of things. There is already evidence that they have been overwhelmed by the response to their platform when you see some of the complaints from developers about insufficient support and poor platform documentation.

In addition, it seems the core Facebook application is not seeing enough attention when you consider that there is some fairly obvious functionality that doesn't exist. Specifically, it is quite surprising that Facebook doesn't take advantage of the wisdom of the crowds for powering local recommendations. If I was a college freshman, new employee or some other recently transplanted individual it would be cool for me to plug into my social network to find out where to go for the best chinese food, pizza, or nightclubs in the area.

I suspect that the speculation on blogs like Paul Kedrosky's is right and we'll see Facebook try to raise a bunch of money to fuel growth within the next 12 - 18 months. To reach its full potential, the company needs a lot more resources than it currently has.


 

If you don't read Stevey Yegge's blog, you should. You can consider him to be the new school version of Joel Spolsky especially now that most of Joel's writing is about what's going on at Fog Creek software and random rants about applications he's using. However you should be warned that Stevey writes long posts full of metaphors which often border on allegory.

Consider his most recent post That Old MarshMallow Maze Spell which is an interesting read but full of obfuscation. I actually haven't finished it since it is rather longer than I tend to devote to a single blog post. I've been trying to track down summaries of the post and the best I've gotten so far are some comments about the post on reddit which seem to imply that the allegory is about being burned out due to some death march project at his current employer.

I'm as down with schadenfreude as the next guy but a death march project seems wildly contradictory to Stevey's previous post Good Agile, Bad Agile where he wrote

The basic idea behind project management is that you drive a project to completion. It's an overt process, a shepherding: by dint of leadership, and organization, and sheer force of will, you cause something to happen that wouldn't otherwise have happened on its own.
Project management comes in many flavors, from lightweight to heavyweight, but all flavors share the property that they are external forces acting on an organization.

At Google, projects launch because it's the least-energy state for the system.
...
Anyway, I claimed that launching projects is the natural state that Google's internal ecosystem tends towards, and it's because they pump so much energy into pointing people in that direction. All your needs are taken care of so that you can focus, and as I've described, there are lots of incentives for focusing on things that Google likes.

So launches become an emergent property of the system.

This eliminates the need for a bunch of standard project management ideas and methods: all the ones concerned with dealing with slackers, calling bluffs on estimates, forcing people to come to consensus on shared design issues, and so on. You don't need "war team meetings," and you don't need status reports. You don't need them because people are already incented to do the right things and to work together well.

So, did anyone else get anything else out of Stevey's post besides "even at Google we have death marches that suck the soul out of you"? After all, I just kinda assumed that came with the territory.


 

Robert Scoble breaks the news that Google brings developers offline with "Gears" where he writes

Right now in Sydney, Australia, the first of 10 Google Developer days are starting up and the audience there is hearing about several new initiatives. The most important of which is “Google Gears,” an open source project that will bring offline capabilities to Web Applications — aimed at developers
...
Regarding Gears. It works on Macs, Windows, Linux on IE, Firefox, Opera. Enables versioned offline storage. Extension to HTML/JavaScript.

They are showing me a demo of the new Google Reader using the new Gears plugin. After you load the Gears plugin you get a new icon at the top of your Reader window which enables offline capabilities of Google Reader. They showed how Google Reader then downloaded 2,000 feed items. They took the browser offline and it continued to work great.
...
Gears supports using Adobe’s Apollo and Flash and should support other technologies including Microsoft’s Silverlight.

Gears will be submitted to a standards organization eventually, they said, but want to make sure the technology is rock solid first.

Am I the only one wondering what took them so long? I remember chatting about this in mid-2005 with Joshua Allen, we were both pretty sure it we would see it happen within a year. I guess a year and a half isn't so bad. :)

The bit about standardizing the technology is a nice touch, not that it matters. What matters is that "it doesn't work offline" is no longer a valid criticism for Google's family of Microsoft office knock offs (i.e. Google Docs & Spreadsheets) or any other AJAX/Flash application that competes with a desktop application. Running it through a standards body wouldn't make a significant difference one way or the other to the adoption of the technology for use with Google apps. It may prevent other developers from adopting the technology but I doubt many developers would look this gift horse in the mouth, after all it is freaking offline support for Web apps. 

Welcome to the future.


 

Recently the mainstream media has been running profiles on people who have built businesses worth hundreds of millions of dollars by buying lots of Internet domain names then filling them with Google ads. Last week, CNN Money ran an article entitled The man who owns the Internet which contained the following excerpt

When Ham wants a domain, he leans over and quietly instructs an associate to bid on his behalf. He likes wedding names, so his guy lifts the white paddle and snags Weddingcatering.com for $10,000. Greeting.com is not nearly as good as the plural Greetings.com, but Ham grabs it anyway, for $350,000. Ham is a devout Christian, and he spends $31,000 to add Christianrock.com to his collection, which already includes God.com and Satan.com. When it's all over, Ham strolls to the table near the exit and writes a check for $650,000. It's a cheap afternoon.
...
Trained as a family doctor, he put off medicine after discovering the riches of the Web. Since 2000 he has quietly cobbled together a portfolio of some 300,000 domains that, combined with several other ventures, generate an estimated $70 million a year in revenue. (Like all his financial details, Ham would neither confirm nor deny this figure.)
...
And what few people know is that he's also the man behind the domain world's latest scheme: profiting from traffic generated by the millions of people who mistakenly type ".cm" instead of ".com" at the end of a domain name. Try it with almost any name you can think of -- Beer.cm, Newyorktimes.cm, even Anyname.cm -- and you'll land on a page called Agoga.com, a site filled with ads served up by Yahoo

The New York Times has a profile on another multimillion dollar company in the same business in today's article entitled Millions of Addresses and Thousands of Sites, All Leading to One which contains the following excerpts

What Internet business has raised $120 million in financing in the last year, owns 725,000 Web sites, and has as its chief executive the former head of Primedia and International Data Group? If you guessed NameMedia, a privately held owner and developer of Web sites based in Waltham, Mass., you take the prize.
...
“What we’ve wanted to do, quietly, is amass the largest real estate position on the Internet, which we feel we have,” Mr. Conlin said. Some of those properties, he said, are the equivalent of “oceanfront” sites, or high-value addresses like Photography.com or DailyHoroscope.com that NameMedia will populate with relevant editorial content. Those who type in any of NameMedia’s other 6,000 or so photography-related Internet addresses, like photographyproducts.com, will land on Photography.com.
...
So far the company’s strategy is paying off, Mr. Conlin said, with company revenue doubling last year, to $60 million.

Companies like this are bad for the Internet for several reasons. For one, they artificially reduce the pool of domain which has resulted in legitimate domains having to choose name that are either awful misspellings or sound like they were stolen from Star Wars. Secondly, a lot of these sites tend to clog up search results especially when they have generic domain names and a couple thousand sites all linking or redirecting back to one domain. Finally, the fact that these companies are making so much money in a manner that is user-hostile and ethically questionable encourages more such businesses which prey on naive Internet users to be formed.

What I've found most shocking about this trend is that the big Web advertising companies like Google go out of their way to court these businesses. In fact, Google has a service called Google AdSense for Domains [with the tastefully chosen URL http://www.google.com/domainpark] which caters exclusively to these kinds of sites.

One of the things I've disliked about the rush towards advertising based business models on the Web is that if unchecked it leads to user-hostile behavior in the quest to satisfy the bottom line. The recent flap over Google and Dell installing the equivalent of spyware on new PCs to show users ads when they make a typo when browsing the Web is an example of this negative trend. Now it turns out that Google is in bed with domain name squatters. These are all examples of Google's Strategy Tax, the fact that they make their money from ads compromises their integrity when there is a conflict between between doing what's best for users and doing what's best for advertisers.

Do no evil. It's now Search, Ads and Apps


 

From the Microsoft press release entitled Microsoft to Acquire aQuantive, Inc we learn

REDMOND, Wash. — May 18, 2007 — Microsoft Corp. today announced it will acquire aQuantive, Inc., for $66.50 per share in an all-cash transaction valued at approximately $6 billion. This deal expands upon the Company’s previously outlined vision to provide the advertising industry with a world class, Internet-wide advertising platform, as well as a set of tools and services that help its constituents generate the highest possible return on their advertising investments.
...
The aQuantive acquisition enables Microsoft to strengthen relationships with advertisers, agencies and publishers by enhancing the Company’s world-class advertising platforms and services beyond its current capabilities to serve MSN. The acquisition also provides Microsoft increased depth in building and supporting next generation advertising solutions and environments such as cross media planning, video-on-demand and IPTV.
...
aQuantive, which has approximately 2600 employees, will continue to operate from its Seattle headquarters as part of Microsoft’s Online Services Business. The combination of Microsoft and aQuantive takes the Company’s advertising platform to the next level in its ability to serve Microsoft’s first party audience assets like MSN, Windows Live, Xbox Live, and Office Live, as well as for third party publishers and applications such as Facebook and Activision game titles.

The first thing I thought when I heard this news is that it is quite telling that Microsoft's biggest acquisition ever is for a Web advertising company. The second is that it seems that Google's overpaying for market share disease ($3.1 billion for DoubleClick and $1.65 billion for YouTube) is contagious. And finally, Microsoft now has offices in downtown Seattle. Oh. Yeah. :)

I worked with some of the Avenue A/Razorfish folks as part of the Social Networking feature in Windows Live Spaces. They seemed like good folks. I'm glad Microsoft has decided to open the pocketbook instead of just rolling over when it comes to buy versus build. Sometimes you really just have to suck it up and buy.

Excerpts from the conference call at TechCrunch.


 

Robert Scoble has a blog post entitled Google to Yahoo and Microsoft: the $1.65 billion was worth it which contains the following excerpt

Ahh, now you all understand what I meant when I said YouTube is a moat, not a revenue generator. By putting YouTube results into Google’s main engine Google ensures it will have better searches than Yahoo and Microsoft (who were, truth be told, getting damn close to matching Google’s quality). And it does it in a way that Yahoo and Microsoft will not be willing to match. Seriously, can you see an executive at Microsoft advocating putting YouTube videos into Microsoft’s search results? I can’t.
...
Anyway, Google just distanced themselves from Yahoo and Microsoft. And they just provided a way to monetize YouTube videos.

I love Google’s strategy. It continues to mess with Microsoft’s strategy. Microsoft still treats each team as something that must make money. Google doesn’t do that. They didn’t care one bit that YouTube didn’t have any revenues. They knew that there’s other ways to make money off of YouTube than to force YouTube to monetize on its own.

Interesting analysis, too bad it doesn't pass muster when you look at the facts. So let's do that
  1. Google didn't need to spend $1.65B on YouTube to integrate their search results. An existence proof of this are video search startups like Blinkx and Dabble that index video from practically every major video source on the Web including YouTube and definitely didn't spend billions doing so. Secondly, Google has already announced that they'll index videos from sites they don't own. Does Robert believe they plan to buy every site whose video content they'll index?

  2. If Google doesn't care about monetizing YouTube why did the company transfer Shashi Seth to YouTube to work on monetization and what about the ads within videos they prototyped a few days ago?

A better analysis of Google Universal Search is that it is the ultimate manifestation of the Features, Not Products initiative. Google had too many search verticals with no way for users to find them (I used to do Google searches for Google Music Search before I could use it) and now they've remedied that in one fell swoop. Just look at what  their users had to deal with before they changed the search engine results page.

In case you were wondering the even more link goes here


 

May 17, 2007
@ 12:16 AM

Danny Sullivan over at Search Engine Land reports on the Google Searchology Day in his entry Google Searchology Day: Recap Of Announcements where he writes about one of the new announcements called "Universal Search" which is excerpted below.

Universal Search: Do a "regular" search and books, news, video, image and local results blended into one single page. It will still mostly be web pages that dominate, but the verticals will be more noticeable. See our very long article detailing this, Google 2.0: Google Universal Search.

So I tried the query bon jovi always and got the following results page.

Notice how there is no tab for music search results even though they are showed inline as integrated results? This seems like a bug given that it works as expected for other queries like virginia tech (which shows a tab for "news" which is also shown inline) and britney spears (which has tabs for six categories including music)  Interesting...

 

A few years ago Joel Spolsky wrote a widely quoted blog post which stated

A very senior Microsoft developer who moved to Google told me that Google works and thinks at a higher level of abstraction than Microsoft. "Google uses Bayesian filtering the way Microsoft uses the if statement," he said. That's true. Google also uses full-text-search-of-the-entire-Internet the way Microsoft uses little tables that list what error IDs correspond to which help text. Look at how Google does spell checking: it's not based on dictionaries; it's based on word usage statistics of the entire Internet, which is why Google knows how to correct my name, misspelled, and Microsoft Word doesn't.

This morning I fired up my favorite RSS reader and saw the danah boyd's entry entitled quality of Google searches? where she pointed out

It's also annoying that they've stopped correcting my atrocious spelling. I mean, it's all fine and well that lots of people in the blogosphere can't spell in exactly the same way that i can't spell, but the #1 type of search i do everyday is spell check. I throw something god-awful like Cziskentmihalyi into the engine knowing that it'll return Csikszentmihalyi. This still works quite well for names but it's stopped working for lots of regular words that i just can't spell to save my life. How pathetic is it that i've started opening up Word for the little red squigglies instead of relying on search? Or maybe both practices are weird...

This seems like a predictable problem. There are lots of commonly misspelled words and in many online communities people have simply given up on correct spelling (heck, I've now grown used to the fact that computer geeks have decided that the correct spelling of ridiculous is rediculous). Thus it is quite likely that a frequently misspelled word eventually occurs so much in the wild that it is considered a valid word. Maybe Google needs some if statements in their code after all, instead of blindly trusting the popularity contest that is Bayesian analysis. :)

The interesting thing is that one could argue that if a particular spelling of a word becomes popular then it automatically is "correct" since that is how the English language has evolved over time anyway.


 

If like me your head is spinning from all the announcements of proprietary "AJAX killers" out there then you probably were still too busy trying to digest Silverlight to even have heard that Sun's announced it's own "AJAX killer" and Flash competitor named Java FX.

Don Park has a good post which serves as a good starting point to grok the meat of this announcement titled Inside JavaFX where he writes

So what is JavaFX? This is what it looks like:

import javafx.ui.*;

Frame {
  title: "Hello World JavaFX"
  width: 20
  height: 50
  content: Label {
    text: "Hello World"
  }
  visible: true
}

As you can see, JavaFX script (*.fx) is basically JSON-like version of XAML which can be either compiled into Java bytecode or run as is using a java-based JavaFX player. Since JavaFX is just java, it can run as applets in browsers, midlets on mobiles, and applications on WebStart.

Sam Ruby has a good follow up to Don's post titled JavaFX Script where he describes some of the innovations and shortcomings of the platform he has noticed in his initial observations.

It looks like Web developers are getting spoiled for choice this summer; Sun vs. Microsoft vs. Adobe vs. Web Standards.


 

By now most people interested in technology news have heard the story initially reported by the New York Post in the article Bill's Hard Drive which claimed that Microsoft was planning to purchase Yahoo! and the subsequent rebuttal of this news in the Wall Street Journal's article Microsoft, Yahoo Discussed Deal. Since I work at Microsoft I'm not supposed to comment on rumors like this so I'll just point to a couple of opinions.

Charlene Li, an analyst at Forrester Research, wrote in her blog post entitled Why Microsoft + Yahoo! makes sense – and why it won’t work that

But there is one major reason why I don’t think Microsoft executives have the stomach for any sort of brand rationalization -- the continued dual branding of Windows Live and MSN. Each time I have a conversation with Microsoft about Windows Live, I get a different explanation of what it is and how it fits with MSN. If the company can’t event figure out its branding strategy with existing properties, I don’t hold out much faith that they could do so with a premium brand like Yahoo!

Robert Scoble writes in his post No “Microhoo”

I’ll tell ya one thing, though. It sure made for interesting conversation with a Yahoo employee I met tonight. We were starting to draw up where value would be built and where it’d be destroyed. There were a lot of places it would be destroyed and if two guys drinking beer can figure that out in half an hour, I’m sure that caused smarter people than us to put on the brakes. For instance one of Yahoo’s biggest properties is its email service. But that’s one of Microsoft’s biggest properties too (aka Hotmail). Then you look at finance sites. Microsoft is doing pretty well there with Microsoft Money (I met an employee who works there too and he said both Microsoft and Yahoo are way ahead of Google in finance traffic). Then you look at mapping. Again, they are pretty strong competitors there. Search? Who’s search technology would be thrown away? Advertising technology? Both Microsoft and Yahoo are pretty close there. Flickr? Clear value creation in an acquisition cause Microsoft doesn’t have anything like it. Same with Del.icio.us, Upcoming.org, Yahoo Answers, MyBlogLog. Portal? Yahoo is clear winner in brand name, but that’s just cause Microsoft has done an awful job in rebranding everything as “Live.”

As for me, two phrases popped into my head when I heard the rumor; hail mary play and post merger layoffs. I guess word association is mother...

In related news, Mike Arrington reported yesterday that Yahoo To Shut Down Yahoo Photos In Favor Of Flickr. This is a shockingly disruptive move that will definitely piss off users given how different the features and target audience of both sites are. Why would a Web savvy, user-centric company like Yahoo! do something that will so obviously cause bad blood between them and their users? How about all the employees working on Yahoo! Photos who busted their butts to make it one of the Web's most popular photo sites? The answer is simple. Yahoo! has redundant and competing products which they could no longer rationalize.

Now go back and reread Robert Scoble's post linked above then consider what a YHOO-MSFT merger would look like again.  


 

April 29, 2007
@ 11:50 AM

Terry Semel, Yahoo! CEO, can't seem to get a break these days. It seems everyone has assumed his days are numbered and he will be ousted for "incompetence". I've seen everyone get into the act from the technology press like C|Net and TechCrunch to more laid back technology pundits like Jason Fried in his post I'd rather be Microsoft than Yahoo.

All of these calls for Terry Semel's head remind me of a book I've been planning to read, The Halo Effect: ... and the Eight Other Business Delusions That Deceive Managers which describes The Halo Effect as it applies to business. So what exactly is this effect? Well according to Wikipedia

The halo effect refers to a cognitive bias whereby the perception of a particular trait is influenced by the perception of the former traits in a sequence of interpretations. The halo effect is involved in Kelley's implicit personality theory, where the first traits we recognize in other people then influence the interpretation and perception of latter ones (because of our expectations). Attractive people are often judged as having a more desirable personality and more skills than someone of average appearance. Celebrities are used to endorse products that they have no expertise in evaluating.

That may have been a bit difficult to follow, so here's an excerpt from David Wolfe's review of the book that explains the Halo Effect in business

Rosenzweig argues that media is no less error-prone in divining the reasons for a company's performance than gurus who write books on company performance.

He uses two prominent case histories to support the latter claim. Cisco and ABB. He recalls for readers how before the dotcom bubble collapse Cisco's John Chambers was widely regarded as the world's best manager and Cisco itself as without surpass in its organizational structure and corporate culture. Beyond that, according to the common view expressed in media, no company operated with greater customer centricity.

Within months of the dotcom collapse the same media that had virtually canonized Chambers were ripping his reputation as a great CEO to shreds. Cisco's was criticized for lack of attention to customer needs. This was alleged to have played a major role in its downturn in revenues and consequent precipitous loss in stock value. Writers told how the company's organization was fragmented and its culture grossly defective.

Rosenzweig tells a similar story of the Swiss-Swedish power company ABB. From being one of Europe's most highly regarded company to being a favorite whipping boy in media within not very many months, ABB turned out to be another example of what Rosenzweig calls the Halo Effect.

As Rosenzweig explains it, the Halo Effect refers to the aura surrounding a company and its leadership that promotes gross generalizations about its nature. When a company is outperforming, most everyone assumes everything about the company is exemplary. When the same company is underperforming, most everyone assumes everything about the company is defective.
Rosenzweig reminds us that externalities accounted mainly for Cisco's and ABB's decline in fortune. Cisco's downturn for example was inextricably linked to its Internet customers who flamed out in droves when the dotcom bubble burst. No amount of executive brilliance could have stayed the fiscal injury inflected by the dotcom bust.

What does this have to do with Yahoo! and Terry Semel? Glad you asked. See the blog post from ValleyWag entitled Fred Vogelstein's Amnesia which contains the following excerpts

How Yahoo blew it. Semel has been Yahoo's CEO for nearly six years, yet he has never acquired an intuitive sense of the company's plumbing. Semel's vaunted dealmaking skills seem to have deserted him. At Yahoo, the marketers rule, and at Google the engineers rule. And for that, Yahoo is finally paying the price. [Fred Vogelstein, writing in Wired Magazine, January 2007]

By figuring out how to make brand advertising work online, Terry Semel is on the verge of creating the 21st century's first media giant. So far, Semel has put together one of the web's hottest winning streaks... It turns out that the riddle of how to succeed online isn't so tough after all. Semel is taking everything he learned in his analog past and marrying it to what he can see in the digital future. [Fred Vogelstein, writing in Fortune Magazine, August 2005]

Same leadership style, same company. Yet when you tweak the externalities the CEO goes from being "on the verge of creating the 21st century's first media giant" to being someone whose "vaunted dealmaking skills seem to have deserted him". From where I sit, it is unclear that Terry Semel is actually doing a bad job, it just seems that his company isn't doing as well as Google which may or may not have anything to do with him.

I wonder how many years it will take until 20% time, free lunches and all the other things that people hype up about Google are used as examples of why the company has issues when some new upstart shows up on the scene?


 

Mike Arrington of TechCrunch has a blog post entitled Can PhotoBucket Survive Without MySpace? where he writes

There was a lot of fingerpointing, denials, and “he said, she said” going on today as everyone digested the news that MySpace had blocked PhotoBucket’s 40 million members from embedding videos into their MySpace pages.

From my perspective this looks like MySpace just found an excuse to send a big middle finger to the largest independent widget company in the hope of disrupting their ongoing acquisition talks. Om Malik sees things differently and thinks Photobucket practically asked for this blockade. Robert Scoble calls Photobucket and services like it “parasitic.” Nick Carr says this is all basically inevitable, regardless of who’s to blame.
...
And many MySpace/Photobucket users will simply leave MySpace and go to one of its many competitors rather than lose the ability to embed their Photobucket media. Re-creating a profile at another social network takes a lot less time than re-uploading hours of video. In the end, Photobucket could prove to be stickier than MySpace.

I agree with Om Malik that PhotoBucket clearly violated MySpace's terms of service by having ads in videos embedded on  http://www.myspace.com. I also think that the advice that Mike Arrington gives in his post will lead PhotoBucket down a bad road if they take it. The key question is whether the lock-in from a social network site like MySpace (where all your friends are) is more significant than the lock-in from having my media stored in a particular photo hosting or video hosting site. If MySpace has only blocked new embeds from PhotoBucket then I'm willing to bet that it is more likely that users will simply pick a new media hosting provider (e.g. YouTube for videos, Flickr for photos) than that they'll switch to Facebook or Windows Live Spaces because they are too tied to PhotoBucket.

If I were PhotoBucket, I'd work with MySpace and either (i) agree on how MySpace gets a revshare of PhotoBucket ads shown on their site or (ii) make it easy for MySpace to filter out the embeds with ads (which are a minority) and allow other embedded media from PhotoBucket pass through. Considering that they are trying to flip their startup, the PhotoBucket crew would be wise to avoid going to war with the biggest traffic driver to their site.


 

The combination of my messed up internal clock and the fact that it seems there is now a wireless LAN at the villa means that I am up blogging when I should be sleeping. The Web geek blog buzz today is the announcement contained in the blog post entitled Map-making: So easy a caveman could do it on the Official Google blog which states

That's why we're announcing My Maps, a new feature that makes it quick and easy to create your own custom Google Maps just by pointing and clicking. You can add placemarks, draw lines and shapes, and embed text, photos and videos -- all using a simple drag and drop interface. Your map automatically gets a public URL that you can share with your friends and family, or you can also publish your map for inclusion in Google Maps search results. We'll continue to show organic local search results with red pushpins; user-generated results will have blue pushpins. The user-created results include KML as well as maps made through My Maps.

To give you a better idea of what kind of maps you can make, here are some examples that Googlers created after we released the feature internally. (We ran a contest and gave a Nintendo Wii to the best map-maker.)

As usual the reactions from the blog pundits are equal parts surprising and unsurprising to me. The unsurprising bit is that I didn't find anyone who compared this to the collections feature of MSN Virtual Earth Windows Live Local Live Search Maps Live Maps which can be viewed at http://collections.live.com. I'm sure when the "Web 2.0" pundits eventually discover we have this feature it will be claimed we copied it from Google. :)

On the other hand what I did find surprising were blog posts like Google Launches MyMaps - Platial Gets Screwed and My Maps at Google: Is Google Doing a Microsoft? from Pete Cashmore and Paul Kedrosky which complained that Google was killing "social mapping" startups like Platial and Frappr with this move. Ignoring that "social mapping" seems like a silly product category in the first place, I wonder what exactly is wrong with this move. Some startups point out consumer demand for certain features from online mapping sites (i.e. missing features) and the consumer mapping sites add the features.

Is the expectation that companies shouldn't be able to improve their products once startups start trying to build a business out of the fact that their product is missing features? I've never understood that reasoning. Paul Kedrosky specifically calls this "pulling a Microsoft". I wonder...do users really want their computers to ship without a Web browser or a media player? Do users really want to go to one site to get maps and another to add annotations to these maps? Adding features to products is good for users and we shouldn't be trying to artificially limit products because some startups are trying to build a business out of a feature. If startups like  Platial and Frappr are actually adding value then they'll survive. If they don't, then they probably didn't have much of a business to begin with.  

PS: I understand that the philosophy of anti-trust law in Europe is about preventing unfair competition whereas in the U.S. it is about preventing harm to consumers. Thus depending on where you are from these questions will strike a different chord with you.


 

I just arrived at London Heathrow and can look forward to another 9 hours or so until my flight to Nigeria. In the meantime, I've found complimentary Web access in the business class lounge so it looks like I won't be bored. I am a little worried about keyloggers and spyware on this computer given how easy it was for me to install Firefox on it. Here are a couple of quick thoughts I had on the way related to links I've seen over the past 24 hours

I'm hungry and need to get back to my book. Holla at y'all later.
 

Mike Arrington has a blog post on TechCrunch entitled Amazon’s War on Statsaholic where he writes

Statsaholic (formerly Alexaholic) launched a year ago and provided much easier access to Alexa traffic data than the Alexa site itself. Statsaholic also had other features Alexa didn’t offer, like embeddable graphs and data smoothing. Others agreed, and soon started linking to Statsaholic instead of Alexa when doing traffic comparisons. At one point, Alexa was the no. 3 search result on Google for “Alexa.”

Statsaholic was not using the Alexa web service to get the data, because Alexa doesn’t offer the graph data via their web service. Amazon, which owns Alexa, could have complained or simply shut them down when it launched, but they didn’t. They actually complimented the service in a post on the Alexa blog last April.
...
What bothers me about the situation is that Amazon sat on it for a year, complimenting the service along the way (and copying it). Then, just when the service started getting really popular, they took drastic measures to shut it down.

I'm totally perplexed by Arrington's position here. Statsaholic is screenscraping Alexa and building a business on top of that. It seems like a pretty open and shut issue to me. The fact that Amazon didn't bother prosecuting them until they got a lot of traffic just points out that there is little point harassing folks who are abusing your service unless they are consuming a lot of your resources or are taking money out of your pocket. It seems Statsaholic was doing both. 

You'd think a smart startup founder would know better than to build a business model on hotlinking and bandwidth theft. You'd also expect a Web savvy dude like Mike Arrington to know better than blame the victim in such situations. Next thing you know, he'll be flaming websites that block hotlinking to their images via htaccess. Weird. 

PS: Someone in the comments wondered how Mike Arrington would feel if someone created a mashup that showed all of the TechCrunch content minus the obnoxious ads (e.g. http://techcrunchminusads.com). I wonder if Mike would sue if the site started stealing a bunch of his traffic  since it wouldn't load so many ads thus being faster and perhaps included more useful info (e.g. crosslinking TechCrunch posts with PodTech interviews)? 


 

The Yahoo! Developer blog has an entry entitled Introducing the Yahoo! Mail Web Service which states

While we are certainly proud of the success of Yahoo! Mail, today we are announcing how we are going beyond the boundaries of the Yahoo! network and enabling developers to build new tools or applications around Yahoo! Mail. We are thrilled to announce the open availability of the Yahoo! Mail Web Service, web service for Yahoo! Mail (accessible via SOAP or JSON-RPC) that we previewed to Yahoo! Hack Day attendees. With the Yahoo! Mail Web Service, you can connect to the core mail platform to perform typical mailbox tasks for premium users such as list messages and folders, and compose and send messages (you can also build mail preview tools for free users with limited Web Service functionality). In other words, developers outside of Yahoo! can now build mail tools or applications on the same infrastructure we use to build the highly-scaled Yahoo! Mail service that serves nearly 250 million Yahoo! Mail users today -- users who might want to help you make some cash with your application.
...
The Yahoo! Mail Web Service is a big release for Yahoo! and the Internet, and it's only the beginning of what you'll be seeing from Yahoo!. Jump into our code samples for Java, .NET, PHP, Perl and Python, and build your dream mail app today, then be sure to give us feedback on your experience so we can continue to make the API even better. Be sure to leverage the online Yahoo! Mail Web Service support group where you can get help from the Yahoo! Mail Web Service team and your fellow developers. We can't wait to see what applications you will build when you add your imagination to the platform. Maybe you want to build an application that backs up Yahoo! mail targeted at a large number of Yahoo! users, or maybe you just want to add a niche feature that makes Yahoo! Mail better for your mom. For inspiration, we've gathered a few applications:

First things first, this is an unprecedented and very welcome move on the part of Yahoo! This is another example of why I think of the three major Web portals, Yahoo! has done the best job of turning their services into a developer platform. I like the fact that the Web service is exposed over multiple protocols and the code samples are in multiple programming languages that run the gamut from enterprise developer fare (Java/.NET) to Web geek fare (Perl/Python/PHP). Mad props to the developer platform folks at Yahoo!, good work.

With that out of the way, there is some stuff that has me scratching my head after taking a look at the Yahoo! Mail Web Service User Guide and API Reference. The first thing that is weird is that although Yahoo! provides SOAP and RESTful JSON web services for accessing one's mail, I still can't get POP access to my Yahoo! mail without shelling out $20/year. After all, GMail has POP access for free and users of the free versions of Windows Live Hotmail can get POP-like access if they use Windows Live Mail desktop although they are restricted to using one mail client.

So I decided to see if the Yahoo! Mail Web Services provides a way around this restriction but found out from the section on "Determining the Users Account Capabilities" that

The Yahoo! Mail Web Service limits the functionality available to free accounts. Premium accounts have no such limits. First call the GetUserData method to get the user’s account type from the web service.
Calls to other APIs will return an error.

So it looks like I actually can't build an application that can be used to read the actual mail messages from my Yahoo! Mail account with the API unless I'm a premium user. Otherwise, all I can do is list the messages but not actually get their content. That makes the APIs a lot less cool than I initially thought. 

Like Sam Ruby and Paul Downey I initially wondered about the choice of exposing a SOAP API but then realized that it may be that they already use SOAP internally so this wasn't that much work for them in that case. I also wondered about the lack of a RESTful non-SOAP XML interface as well but after looking at the complex object models I can see why they went with data formats that are pretty much serialized object models (i.e. Javascript OBJECT Notation & Simple OBJECT Access Protocol) instead of expecting developers to write a bunch of gnarly XML parsing code for processing over a dozen different response formats from the 23 methods in the Yahoo! Mail Web Service.

I suspect that Yahoo! won't get as much traction as they expect with the API until they remove some of the restrictions on non-premium accounts. Even then it does look like there is enough to build Windows and Windows Live gadgets for Yahoo! Mail that show your messages. Except that there is no way to read the mail contents and not even a way to link to the message in way that sends the user to Yahoo! Mail to read its contents. I bet if Yahoo! fixed the latter and perhaps had a rev-share with people who drove clicks back to the Web-based mail client, things would get very interesting. I wonder if Jeremy is listening?


 

The Australian iTWire has a rather biased and misinformed article entitled Official: Microsoft ‘bribes’ companies to use Live Search which contains the following excerpt

Microsoft’s new program is called “Microsoft Service Credits for Web Search” and has been unveiled by John Batelle’s ‘SearchBlog’. The money on offer is significant, especially when multiplied across thousands of PCs. The deal means that companies can earn between US $2 and US $10 per computer on an annual basis, plus a US $25,000 “enrollment credit” which is a nice big wad of cash that will likely need a large-ish, strong and sturdy brown paper bag to hold securely while being passed under the table.  

For companies that have thousands of computers, this could translate into anywhere from US $100,000 to $200,000 per year, which is money that could be put to good use in the IT department or elsewhere in the company.
...
Former Microsoft employee and blogger Robert Scoble who served as the online face of Microsoft during his three years at the company is not impressed with Microsoft’s moves in deciding to offer companies money to use search.  His arguments are quite valid and boil down to Microsoft really needing to create better products, rather than needing to pay companies to get more traction for Windows Live. After all, Scoble isn’t the first to observe that Google doesn’t need to pay anyone to use its search services – people use them voluntarily because of the quality of the results

The amount of bias in this article is pretty amazing considering that Microsoft is primarily reacting to industry practices created by the Google [which have also been adopted by Yahoo!]. Let me count the ways Google bribes companies and individuals to use their search engine

  1. Google pays AdSense publishers for each user they convince to install Firefox with the Google Toolbar installed. Details are in the documentation for the AdSense Referrals Feature. Speculation on Slashdot was that they pay $1 per user who switches to Firefox + Google Toolbar.

  2. Google paid Dell $1 billion dollars to ensure that Google products are preinstalled in all the computers they sell and the default search engine/home page is set to Google. Details of this deal were even published in iTWire.

  3. Google paid Adobe an undisclosed amount to bundle Google Toolbar [which converts your default search engine in your browser to theirs] with all Adobe products.

  4. Google entered a distribution deal with Sun Microsystems to bundle Google Toolbar [which converts your default search engine in your browser to theirs] with all new installations of the Java runtime.

  5. Google products which converts your default search engine in your browser to theirs are bundled with the Winzip archiving utility. Financial details of the deal were undisclosed.

  6. Google is the default search engine for both the Opera and Firefox browsers. Both vendors get a cut of the search revenue generated from user searches which runs in the millions of dollars.

I could go on but my girlfriend just told me it's time for breakfast and I'm already in trouble for blogging on a Sunday morning. However the above links should be enough to debunk the inaccurate statements in the iTWire article. I guess iTWire's "journalism" is further validation of the saying that you should never let the facts get in the way of a good flame.


 

A number of blogs I read have been talking about Amazon's S3 service a lot recently. I've seen posts from Jeff Atwood, Shelley Powers and most recently Dave Winer. I find it interesting that S3 is turning into a classic long tail service that works for both startups who are spending hundreds of thousands to millions of dollars a year to service millions of users (like Smugmug) to bloggers who need some additional hosting for their cat pictures. One reason I find this interesting is that it is unclear to me S3 is a business that will be profitable in the long term by itself.

My initial assumption was that S3 was a way for Amazon to turn a lemons into lemonade with regards to bandwidth costs. Big companies like Amazon are usually billed for bandwidth using 95th percentile billing, which is explained below

With 95th percentile billing, buyers are billed each month at a fixed price multiplied by the peak traffic level, regardless of how much is used the rest of the time. Thus with the same nominal price, the effective price is higher for buyers with burstier traffic patterns.

So my assumption was that S3 allows Amazon to make money from bandwidth they were already being charged for and not using. As for storage, my guess is that they are either making a miniscule amount of profit or at cost. Where this gets tricky is that, if S3 gets popular enough then all of a sudden it no longer is a way to make money from bandwidth they are being billed for but aren't using but instead impacts their actual bandwidth costs which then changes the profit equation for the service. Without any data on Amazon's cost structure it is unclear whether this would make the service unprofitable or whether this is already factored into their pricing.

On the other hand, Amazon's Elastic Compute Cloud (EC2) isn't something I've seen a lot of bloggers rave about. However it seems to be the service that shows that Amazon is making a big play to be the world's operating system in the sky as opposed to dabbling in providing some of its internal services to external folks as a cost savings measure. With EC2 you can create a bunch of virtual servers in their system and load it up with an Amazon Machine Image (AMI). An AMI is basically a server operating system and the platform components you need on it. Typical AMIs are an instance of a LAMP system (Linux/Apache/MySQL/PHP/Perl/Python) although I did see one AMI that was an instance of Windows 2003 server. You can create as many or as few server instances as you need and are billed just for what you need.

I suspect that the combination of EC2 and S3  is intended to be very attractive to startups. Instead of spending hundreds of thousands of dollars building out clusters of servers, you just pay as you go when you get your monthly bill. There are only two problems with this strategy that I can see. The first is that, if I was building the next Digg, Flickr or del.icio.us I'm not sure I'd want to place myself completely at the mercy of Amazon especially since there doesn't seem to be any SLA published on the site. According to the CEO of Smugmug in his post Amazon S3: Outages, slowdowns, and problems they've had four major problems with S3 in the past year which has made them rely less on the service for critical needs. The second issue is that VC money is really, really, really easy to come by these days judging from the kind of companies that get profiled on TechCrunch and Mashable. If the latter should change, it isn’t hard to imagine dozens of enterprising folks with a couple of thousand dollars in their pockets deciding to go with S3  + EC2 instead of seeking VC funding. But for now, I doubt that this will be the case.  

What I suspect is that without some catalyst (e.g. the next YouTube is built on S3  + EC2)these services will not reach their full potential. This would be unfortunate because I think in much the same way we moved from everyone rolling their own software to shrinkwrapped software, we will need to move to shrinkwrapped Web platforms in the future instead of everyone running their own ad-hoc cluster of Windows or LAMP servers and solving the same problems that others have solved thousands of times already.

I wonder if Amazon has considered tapping the long tail by going up against GoDaddy's hosting services with S3  + EC2. They have the major pieces already although it seems that their prices would need to go down to compete with what GoDaddy charges for bandwidth although I suspect that Amazon's quality of service would be better.


 

Today on the Facebook blog I spotted a post entitled FQL which contains the following excerpt

Two and a half months ago, a few of us were hanging out in the Facebook TV room, laying on the Fatboys and geeking out about how to move forward with the API for the Facebook Platform. We had a beta version that was fully functional, but we kept wishing that the interface were cleaner, more concise, and more consistent. Suddenly it occurred to me – this problem had been solved over 30 years earlier by database developers who came up with SQL – the Structured Query Language. What if we could use the same time-tested interface as the way for developers to access Facebook's data?
...
This isn't a simple problem – with millions of users and billions of friend connections, photos, tags, etc., Facebook's data doesn't exactly fit into your average database. And, even if it did, we still have to carefully apply all of those complicated privacy rules. Facebook Query Language would have to take those SQL-style queries from developers, figure out what data they're actually looking for, figure out if they're allowed to actually see the data, figure out where the data is stored, and then finally go and get the data to return back to the developer. I knew building FQL would be hard, but that's why I couldn't wait to do it.

This is one of those things I used to think was a great idea when I was on the XML team at Microsoft. Instead of exposing your data using APIs, why not expose your data as XML then allow people to perform XQuery operations over the data. In reality, this often isn't really feasible because you don't want people performing arbitrary queries over your data store that may request data too much data (SELECT * FROM blog_posts) or are expensive computationally.

Looking at the FQL developers guide it states that a typical queries look like

SELECT name, pic FROM user WHERE uid=211031 OR uid=4801660

SELECT name, affiliations FROM user
WHERE uid IN (SELECT uid2 FROM friend WHERE uid1=211031)
AND "Facebook" IN affiliations.name AND uid < 10

SELECT src, caption, 1+2*3/4, caption, 10*(20 + 1) FROM photo
WHERE pid IN (SELECT pid FROM photo_tag WHERE subject=211031) AND
pid IN (SELECT pid FROM photo_tag WHERE subject=204686) AND
caption

and return results as XML. I've assumed that what is supported is a simple subset of SQL, perhaps written with Lex & Yacc or ANTLR but it still seems somewhat problematic to move away from the constrained interface of an API and provide access via a query language. It is definitely a lot cooler and more consistent to work with a query language than an API though. Later on when I have some free time, I'll see if I can deduce the grammer for FQL by trying out queries in the Facebook API test console. It looks like there goes one of my evenings this week.

Nice work.


 

From the Microsoft press release Microsoft Demonstrates Further Commitment to Healthcare Market With Planned Acquisition of Web Search Company we learn

NEW ORLEANS — Feb. 26, 2007 — Microsoft Corp. today announced that it has agreed to acquire Medstory Inc., a privately held company based in Foster City, Calif., that develops intelligent Web search technology specifically for health information. The acquisition represents a strategic move for Microsoft in the consumer health search arena and signals a long-term commitment toward the development of a broader consumer health strategy. Medstory employees will join the Health Solutions Group, a recently formed division at Microsoft that will manage product development and delivery. Financial terms were not disclosed, as part of the agreement between the organizations.

This reminds me of the post Thoughts on health care, continued from Google's Adam Bosworth which stated

As I indicated in my post last week, I've been interested in the issue of health care and health information for a while. I just spoke at a conference about some of the challenges in the health care system that we at Google want to tackle. The conference, called Connecting Americans to Their Health Care, is a gathering focused on how consumers are transforming health care through the use of personal health technologies.

This speech will give you some insight into the problems that we believe need our attention.

It is also interesting that Adam Bosworth had been billed with the title Architect, Google Health for a while. I'd once heard that the the market for medical related keywords is one of the most lucrative for search engines which may explain the interest. However if you look at the list of most expensive adwords it would seem that building a vertical search engine targetted at debt consolidation is the real goldmine. :)


 

I'm almost caught up on my blog reading since getting back from vacation and I've spotted a couple of items I'd have blogged responses to if I was around. Since I don't have the time to write full blog posts on each of these items, here are links to the posts and brief outlines on what I thought about them

  • Harish Mallipeddi has a blog post entitled Measuring efficiency of tagging with Entropy links to the paper Understanding Navigability of Social Tagging Systems by Ed Chi and Todd Mytkowicz of Xerox Parc which excerpts the key findings from the paper. One result of their research which seems obvious in hindsight and shows one of the issues that social software has to deal with as its community of users grows was

    The way he does that is to measure entropy (yup that same old same old Claude Shannon’s information theory which you learned in one of the CS courses) of entities like documents (D), users (U) and tags (T). His research group crawled the entire del.icio.us archive and then calculated the entropies. Here’s what they found:

    • H(D|T) specifies the social navigation efficiency. How efficient is it for us to specify a set of tags to find a set of specific documents? We found that in del.icio.us that it is getting less and less efficient.

    This makes sense when you think about it. Let's say the first set of users of del.icio.us came from a homogenous software development background and started applying the tag "xml" to mean items about the eXtensible Markup Language. Later on as the community grew, a number of gamers joined the site and they now use the tag "xml" to refer to items about the game X-Men Legends. Now if you are one of the original geek users of the site, the URL http://del.icio.us/tag/xml no longer is just about markup languages but also about video games. To actually find items strictly about the eXtensible Markup Language you may have to add other tags as refinements such as http://del.icio.us/tag/xml+programming.

    What this means is that to the oldest users of the site, the quality of the tagging system will seem to degrade over time even though this is a natural consequence of growth and diversifying its user base. Of course, this is only a problem if a lot of people use del.icio.us to find all items about a topic (i.e. browsing by tags) as opposed to just storing their individual bookmarks or subscribing to the bookmarks of people they know and trust.

  • It seems Google announced some sort of Microsoft Office killer last week. You can read Don Dodge's Why Microsoft will not fall into the Innovators Dilemma and Robert Scoble's Microsoft has no innovator’s dillema? for two conflicting opinions on how this affects Microsoft. Personally, I think I've overdosed on the amount of times I've read the words innovator's dilemma in association with this announcement while catching up on email and blogs. What is funny about this situation is that almost everyone I've seen who throws the term around doesn't seem to have read the book. It is quite interesting to see Don Dodge write sentences like

    Microsoft will do everything possible to preserve these businesses while transitioning to the new Live strategy.
    and then follow that up with "No Innovators Dilemma here" without seeing the obvious contradiction in his words. Lots of  doublethink at work it seems.

    A side effect of reading this set of blog posts is that I found Don Dodge's Innovate or Imitate...Fame or Fortune? which praises being a fast follower as being more valuable than being an innovator. I've found that a lot of people at Microsoft point to past and recent successes such as XBox, Microsoft Office and Internet Explorer as proof that being a "fast follower" is the best strategy for Microsoft. There are three key problems with this kind of thinking

    1. It assumes your competitors are incompetent. This may have worked in the old days but with competitors like Google and Apple Inc, it isn't the case anymore.
    2. It requires that you have an ace up your sleeve that significantly one ups the competitors when you ship your knock off (e.g. integrating disparate applications into an Office Suite and pricing it lower than competitors, integrating product into the operating system, integrating a rich and social online experience into what was previously a solitary experience etc).
    3. It ignores the fact that "first mover advantage" is actually true for applications that have network effects which is definitely the case for social software which a lot of software has become today.

  • The "diversity in conferences" recurring debate was kicked off again by a blog post by Jason Kottke entitled Gender Diversity at Web Conferences which encouraged the interesting responses from folks like Eric Meyer, Anil Dash and Shelley Powers. They are all good posts with stuff I agree and disagree with in them but I wasn't moved to write until I read the post Why are smart people still stuck on gender and skin-color blinders? by Tantek Çelik where he wrote

    Why is it that gender (and less often race, nay, skin-color, see below) are the only physical characteristics that lots of otherwise smart people appear to chime in support for diversity of?

    E.g. as long as we are trying for greater diversity in superficial physical characteristics (superficial because what do such characteristics have to do with the stated directly relevant criteria of "technical expertise, speaking skills, professional stature, brand appropriateness, and marketability" - though perhaps I can see a tenuous link with "rainbow" marketing), why not ask about other such characteristics?

    Where are all the green-eyed folks?

    Where are all the folks with facial tattoos?

    Where are all the redheads?

    Where are the speakers with non-ear facial piercings?

    Surely such speakers would help with "hipness" marketing.

    I found this post to be disingenious and wondered how anybody could downplay the gender and racial bias in the "Web 2.0" technology conference scene by equating it to a preference for green eyed speakers. So I decided to throw in my $0.02 on this topic...again.

    After the last ETech, I realized I was seeing the same faces and hearing the same things over and over again. More importantly, I noticed that the demographics of the speaker lists for these conferences don't match the software industry as a whole let alone the users who we are supposed to be building the software for.

    There were lots of little bits of ignorance by the speakers and audience which added up in a way that rubbed me wrong. For example, at the 2005  Web 2.0 conference a lot of people were ignorant of Skype except as 'that startup that got a bunch of money from eBay'. Given that there are a significant amount of foreigners in the U.S. software industry who use Skype to keep in touch with folks back home, it was surprising to see so much ignorance about it at a supposedly leading edge technology conference. The same thing goes for how suprised people were by how teenagers used the Web and computers. Additionally, there are just as many women using social software such as photo sharing, instant messaging, social networking, etc as men yet you rarely see their perspectives presented at any of these conferences. 

    When I think of diversity, I expect diversity of perspectives. People's perspectives are often shaped by their background and experiences. When you have a conference about an industry which is filled with people of diverse backgrounds building software for people of diverse backgrounds, it is a disservice to have the conversation and perspectives be homogenous. The software industry isn't just young white males in their mid-20s to mid-30s nor is that the primary demographic of Web users.

    Personally, I've gotten tired of attending conferences where we heard more about technologies and sites that the homogenous demographic of young to middle aged, white, male computer geeks find interesting (e.g. del.icio.us and tagging) and less about what Web users actually use regularly or find interesting (hint: it isn't del.icio.us and it sure as fuck isn't tagging).


 

February 25, 2007
@ 04:37 PM

Twice this week, I've been impressed by how some rant I made in my blog turned into an implemented feature in Web software that hundreds of thousands of people use. The first incident comes from my blog post Why Feedburner Doesn't Count Outlook 2007 Subscribers where I wrote about the fact that FeedBurner doesn't track subscribers to my feed who're using Outlook 2007 because it uses the same user agent string as Internet Explorer 7. Thus I was pleasantly surprised when I logged into FeedBurner and saw the following

It seems that while I was on vacation the folks at FeedBurner decided to implement a solution to the problem I pointed out even though it isn't their fault. Nice.

The second incident comes in response to my post MSN SoapBox in Public Beta where I mentioned that neither Google Reader nor Bloglines would display videos from MSN Soapbox embedded in a blog post. Yesterday Mihai Parparita who works on Google Reader let me know that they added support for that while I was on vacation. That means if you are reading this in Google Reader you should see a video on the next line

Thanks to the Google Reader team for implementing this so quickly.


 

February 12, 2007
@ 07:55 PM

Nick Carr has a blog post entitled Googlegate in North Carolina where he writes

North Carolina's Senate Finance Committee is hastily arranging hearings for next week on the state's use of tax incentives to lure businesses, as public outrage mounts over disclosures that Google was granted as much as a quarter billion dollars in secret tax breaks for a plant expected to employ approximately 200 workers. There's no word yet on whether any Google officials will be asked to testify.
...
The Googlegate controversy is unlikely to abate any time soon. Troubling new details of the secret deal-making continue to emerge. Today's Charlotte Observer features a long article describing how public officials leaned on some local residents to sell their homes to make way for the Google plant. The mayor of the town of Lenoir, Davis Barlow, and the county commissioner, Tim Sanders, were among the officials who, according to the paper, went "door-to-door on behalf of the Internet giant Google. In some cases, officials returned to homes four or five times. Barlow and Sanders effectively used the personal touch, avoiding a drawn-out public debate that Google was secretly telling them would scuttle the deal. That personal touch enabled some residents to feel comfortable in selling their property."

This reminds me of a comment I once heard about why the deal makers at GOOG are such hardball players. It goes back to the Google Founders' Awards which were intended to be a way to significantly reward people who add value to the company's bottom line.  Since this award is worth millions of dollars, there is a lot of stiff competition and I'd heard that it ended up the sales folks, acquisitions experts and other deal makers who end up as the primary contenders for the award.

I guess it makes sense, which other job functions can say that they directly save or benefit the company a quarter of a billion dollars on the bottom line? Not the lead developer of Google Calendar or the PM who wrote the spec for Google Base, that's for sure. :)

Unfortunately, when you put millions of dollars in incentives in front of your employees you shouldn't be surprised if they start cutting ethical corners to make things happen. Even CEOs and CFOs aren't immune from this which is why we have Sarbanes Oxley today.


 

It's unfortunate that when people become successful, there are always the jealous waiting on the wings to tear them down and exaggerate every flaw. Today was one of those days for Flickr. You can tell things are fishy when the people complaining loudest about a company aren't its users but its competitors. We have

  1. There's Thomas Hawk who writes There's Some Mighty Pissed Off Flickr Members Right Now

    [I am CEO of Zooomr]

    Flickr needs to reverse the asinine decisions made today to force people to merge their accounts with Yahoo and to place new limits on your contacts and tags.

    Here are some of the more interesting comments pulled over the past few hours from two flickr forums where they are taking a beating over this right now.

    People are pissed.

  2. And There's Don MacAskill who doesn't just stop at gloating stoking the flames reporting on the problems at a competitor's service. In his post The Dark Side of the Flickr Acquisition where he writes

    You asked for it, you got it: SmugMug is offering 50% off to all Flickr refugees. Just sign up for our free trial using the coupon code flickr and if you like what you see, you’ll get 50% off your first year.

    We’re getting some email from ‘Old Skool’ Flickr users asking us if they can get a discount because Yahoo’s making some changes they don’t like. Thomas Hawk has more coverage over on his blog, you can read the Flickr Forums for more reactions, and even check out the Flick Off group (aka the Flickr Accounts Mass Suicide Countdown group).

I looked at the announced Flickr changes and didn't see anything particularly shocking. For performance/architectural reasons, they are limiting people to having 3000 buddies and 75 tags. That sucks but it isn't the end of the world. If anything, it just points out the architectural limits of Flickr's backend to competitors but those limits should be fine for most users. The second issue seems to be that you'll soon need a Yahoo! ID to access Flickr. As Anil Dash states in his post I am okay with my Yahoo sign-in, people have known this for months so it shouldn't be a surprise and they've encouraging people to get a Yahoo! login on the sign-in page for a while now.

I wouldn't be surprised if a lot of the "aggrieved users" are actually astroturfers from competing Web 2.0 photo sharing sites.


 

I like the concept of online Q&A sites and I have to say that I've been quite impressed at how successful Yahoo! has been with Yahoo! Answers. Not only did they build a good end user experience but they followed up with heavy cross promotion on their other services, TV ads and getting lots of real-world celebrities to use the service. My favorite questions asked by real-world celebrities thus far

Based on your own family's experience, what do you think we should do to improve health care in America? asked by Hillary Clinton (U.S. Senator and Presidential Candidate)

What should we do to free our planet from terrorism? asked by Dr. APJ Abdul Kalam (President of India)

That's pretty freaking cool. Kudos to the Yahoo! Answers for being able to pull off such a great promotion and build such a successful service in such a short time. 


 

January 26, 2007
@ 02:13 AM

Interesting, it seems Flickr have formalized the notion of partitioning tags into namespaces with their introduction of Machine Tags which are described as

# What are machine tags?

Machine tags are tags that use a special syntax to define extra information
about a tag.

Machine tags have a namespace, a predicate and a value. The namespace defines a class or a facet that a tag belongs to ('geo', 'flickr', etc.) The predicate is name of the property for a namespace ('latitude', 'user', etc.) The value is, well, the value.

Like tags, there are no rules for machine tags beyond the syntax to specify the parts of a machine tag. For example, you could tag a photo with :

* flickr:user=straup

* flora:tree=coniferous

* medium:paint=oil

The XML geek in me can't help but squint at the term "namespaces" and wonder how they plan to avoid naming collisions in a global namespace (e.g. if multiple people choose the same name for a namespace they create) . I guess this is no different from people using the same word to tag an item while meaning totally different things (e.g. "apple", "glass", "light", etc) and folksonomies like Flickr seem to handle this just fine.

Creating facets in tags like this isn't new, del.icio.us has had this for a while and it it looks like a good way to create ways to create hidden tags that the system can use for performing special operations without it being in the user's face.

Now that the two granddaddies of tagging both provide this functionality, I wonder how long it takes for machine tags to wind it's way through all the tagging systems in the various copycat Web 2.0 sites on the Web.


 

In response to my recent post entitled ODF vs. OOXML on Wikipedia one of my readers pointed out

Well, many of Weir's points are not about OOXML being a "second", and therefore unnecessary, standard. Many of them, I think, are about how crappy the standard actually is.

Since I don't regularly read Rob Weir's blog this was interesting to me. I wondered why someone who identifies himself as working for IBM on various ODF technical topics would be spending a lot of his time attacking a related standard as opposed to talking about the technology he worked. I assumed my reader was mistaken and decided to subscribe to his feed and see how many of his recent posts were about OOXML. Below is a screenshot of what his feed looks like when I subscribed to it in RSS Bandit a few minutes ago

Of his 24 most recent posts, 16 of them are explicitly about OOXML while 7 of them are about ODF.

Interesting. I wonder why a senior technical guy at IBM is spending more time attacking a technology whose proponents have claimed is not competitive with it instead of talking about the technology he works on? Reading the blogs of Microsoft folks like Raymond Chen, Jensen Harris or Brian Jones you don't see them dedicating two thirds of their blog postings to bash rival products or technologies.

From my perspective as an outsider in this debate it seems to me that OOXML is an overspecified description of an open XML document format that is backwards compatible with the billions of documents produced in Microsoft Office formats over the past decade. On the other hand, ODF is an open XML document format that aims to be a generic format for storing business documents that isn't tied to any one product which still needs some work to do in beefing up the specification in certain areas if interoperability is key.

In an ideal world both of these efforts would be trying to learn from each other. However it seems that for whatever reasons IBM has decided that it would rather that Microsoft failed at its attempt to open up the XML formats behind the most popular office productivity software in the world. How this is a good thing for Microsoft's customers or IBM's is lost on me.

Having a family member who is in politics, I've learned that whenever you see what seems like a religious fundamentalism there usually is a quest for money and/or power behind it. Reading articles such as Reader Beware as ODF News Coverage Increases it seems clear that IBM has a lot of money riding on being first to market with ODF-enabled products while simultaneously encouraging governments to only mandate ODF. The fly in the ointment is that the requirement of most governments is that the document format is open, not that it is ODF. Which explains IBM's unfortunate FUD campaign. 

Usually, I wouldn't care about something like this since this is Big Business and Politics 101, but there was something that Rick Jellife wrote in his post An interesting offer: get paid to contribute to Wikipedia which is excerpted below

So I think there are distinguishing features for OOXML, and one of the more political issues is do we want to encourage and reward MS for taking the step of opening up their file formats, at last?

The last thing I'd personally hate is for this experience to have soured Microsoft from opening up its technologies so I thought I'd throw my hat in the ring at least this once.

PS: It's pretty impressive that a Google search for "ooxml" pulls up a bunch of negative blog posts and the wikipedia article as the first couple of hits. It seems the folks on the Microsoft Office team need to do some SEO to fix that pronto.


 

Categories: Competitors/Web Companies | XML

January 16, 2007
@ 08:23 PM

By now it's common news that Google has been hit by what seems like half a dozen or more cross site scripting security flaws in the past month. If you missed the news, you can read blog posts like More Google security failures and Wow, more Google XSS problems which contain links to some of the stories of recent exploits. The bugs in those blog posts aren't exhaustive, I've seen some blog posts about exploits that don't seem to have hit the mainstream tech blogs such as the one mentioned in the blog post Pending Members - Google Groups XSS Bug [Part 2].

Anyway, the fact that Google is having problems with XSS issues isn't terribly interesting and should be an expected part of the growing pains as they go from a service that doesn't store any user data to one that aims to be the repository of all their user's data. That requires an entirely different approach to security. What I did find interesting was a blog post on the Google Blogoscoped blog entitled On Google Security which stated

Today, it almost seems as if every single product team in the Googleplex has the “power” to accidentally introduce a Google Account risk with an HTML injection hole, or another kind of cross-site scripting issue. An exotic Blogger bug was able to reveal your Google Docs, even if you’re not blogging with Blogger – an improbable Google Base bug was able to reveal your personalized homepage, even when you’ve never worked with Google Base**. I would argue: these things happen, individual developers and developer teams make errors. It’s impossible not to. There are ways to automatically test against HTML injections, but such tools too need to be handled by humans.

The real problem, and solution, might be on the higher level of the system architecture – the way Google integrates its services and handles cookie data. Right now, the Google Office product partly resembles a mighty convenient & long chain... a chain which is only as strong as its weakest link. Is this a trade-off we’ll just have to make with future web apps, or are there ways to improve on the situation... either by users, or those building browsers, or those developing web apps?

Those who ignore history are doomed to repeat it. None of the problems listed are unique to Google. Any portal that provides multiple services that require the user to login is vulnerable to these problems. This includes competing portals like Yahoo!, MSN and AOL. All of these services have had to encounter and protect users against the very same problems Google is having difficulty dealing with today.

It is likely that with time, Google will stumble upon the same set of best practices that are common knowledge amongst its portal competitors who have been in the game a lot longer. Thinking that this is a problem that affects "the future of Web apps" ignores the history of the Web. 

In the meantime, if you are a Web developer at Google, I'd suggest reading Chapter 12 of Writing Secure Code by Michael Howard. After that, take a look at You know about XSS. How about XSRF/CSRF? which happens to use a Google service as an example of Cross Site Request Forgery attack (XSRF).

That which doesn't kill us only makes us stronger. ;)


 

January 16, 2007
@ 05:57 PM

Danny Sullivan over at Search Engine Land has a post entitled comScore: Google Wins Again & IE7 Doesn't Stop Microsoft's Slide where he writes

It's that time again -- search popular stats for last month are coming out. Actually, Hitwise sent me their figures earlier this month but I'm diving in with the comScore figures that just came out. The main real news is despite the Internet Explorer 7 launch, Microsoft's Live continues to show a drop in usage.

What is puzzling to me is that people thought that the release of IE 7 would cause a increase in search share for Microsoft's search engine and a decline in competitors. The fact is that built-in search boxes within the browser encourage people to treat search as a feature of the browser instead of a site they visit. That means that the defaults built into the browser/operating system are important. But what exactly is the default search engine on most PCs running IE 7? I don't have any hard numbers but here's some data from my post about this entitled Competing with Google is Like the War in Iraq which stated

The combination of the proliferation of search toolbars and a new generation of Web browsers with built-in search boxes (e.g. IE 7 and Firefox) have reduced the need for users to actually go to websites to perform a search. This means that it is now very important to be the search engine that is used when a user enters a search directly from their browser. Guess which search engine is the one used by your browser if you
  1. Are you a user of the Firefox browser?
  2. Are you a user of the Opera browser?
  3. Are you a user of IE 7 and have installed Adobe Acrobat?
  4. Are you a user of IE 7 and have installed the Java runtime?
  5. Are you a user of IE 7 and have installed the WinZip archive utility?
  6. Are you using a newly purchased Dell computer?
  7. Are you a user of the Google Toolbar?
Yes, the answer is Google in every case. So even if you are an Internet n00b who hasn't made up their mind about which search engine to choose, there is a large chance that the default search engine you end up using thanks to recent innovations in IE 7 and Firefox will be Google.

If anything, browsers like Firefox and IE 7 make it harder for users to switch from Google not easier because it gets them away from the notion of visiting websites to perform searches and instead they just accept whatever default the browser provides.


 

My sister is paying me a surprise visit this weekend and I decided to look on the Web for ideas for what we could do together this weekend. My initial thoughts were that we'd go to the movies and perhaps check out the Bodies: The Exhibition. I wouldn't to see if I could get a better suggestion on the Web.

My first instinct was to try Seattle - City Search but had to give up when I realized the only events listed for today were either announcements of what DJs would be at local clubs tonight or announcements sales at local stores. Another thing that bugged me is how few ratings there were for events or locations on City Search. This reminds me of a blog post on Search Engine Land entitled Local And The Paradox of Participation which came to a set of incorrect conclusions about a poll that claimed that people are equally likely to post a positive or negative review of an event or location. The incorrect conclusion was that it is a myth that few people are likely to post reviews. Given that locations and events that are attended by thousands of people tend to only have dozens of reviews on almost every review site I've ever seen seems to make this a fact not a myth. The poll only seems to imply that people are willing to share their opinion if prompted which is totally different from someone attending a nightclub or concert then feeling compelled to visit one of umpteen review sites to share their opinion. What is surprising to me is that there doesn't seem to even be a small community of die hard reviewers on City Search which is unlike most review sites I've seen. Just compare Amazon or IMDB which both seem to have a number of reviewers who are on top of certain categories of products.

Anyway, what does this have to do with Google? Well, I went to Rich Skrenta's much vaunted starting point of the Intenet and tried some queries such as "local events", "seattle events" and "events in seattle" with pathetic results. The only useful links in the search results page led me to a couple of event search engines (e.g. NWsource, Upcoming) that were pathetically underpopulated with event information. None of them even had a listing for Bodies: The Exhibition. Lame. 

I tried Google Local which turned out to be redirect to their mapping site. Shouldn't a local search engine be able to find events in my local area? Double lame.

Before you bother pointing it out, I realize that other search engines don't do a much better job either. This seems to point to an opportunity to add a lot of value in what must be a very lucrative search market. I'm surprised that Yahoo! hasn't figured out how to do more with their purchase of Upcoming.org. Then again Yahoo! hasn't figured what to do with any of the Web 2.0 startups they've purchased so maybe that is expecting too much. Maybe Google will purchase Eventful.com and fix this fairly big hole in their search offerings. Somehow I doubt it. .


 

The bigger Google grows, the more it seems that like every large company it's products now have to pay a strategy tax which may end up shortchanging their users. If you are unfamiliar with the term, you should read Dave Winer's excellent essay Strategy Tax which is excerpted below

Ben explained that sometimes products developed inside a company such as Microsoft have to accept constraints that go against competitiveness, or might displease users, in order to further the cause of another product. I recognized the concept but had never heard the term.

An example. Consider a company that develops both a Web browser and a word processor. The product team for the browser might get lots of input from users saying "We'd like a better editor in the browser." It would be natural to give the users what they want. So they put the feature in their project plan, but when it comes up for review, the CEO shoots them down. "They should use the word processor," he says.

Another example. An electronics company makes a portable music player, and also owns a recording studio. The team working on the music player knows that there's a big market for players that work with a format that's popular with computer users, but lacks sophisticated copy protection. They project huge sales. But when the product comes up for review, the CEO shoots them down. "We'll lose sales of our music," he says.

Before I read this essay I thought this was a problem unique to Microsoft and also thought that I was being astute in observing the trend at the company when in truth the term was part of the cultural knowledge of the company while I was still in programming diapers. Over time, it has become clear to me that this is a problem that affects any business endeavor where different product units either rely on each other or indirectly compete with each other [assuming that the company isn't dysfunctional enough to have products that directly compete against each other]. Below are three examples of how the strategy tax is affecting Google, all of which are observations by other bloggers which I've noticed myself but refrained from mentioning since I work for a competitor and it would have come off as sour grapes.

Disincentive to Improve Search Due to Ad Business

In his post Good Luck Jimmy, Dave Winer writes

Google is repeating the pattern of the previous generation of search engines (Alta Vista, Infoseek) were doing when Google zigged to their zag, so successfully. Today, Google is fattening up and spreading out, going after Microsoft in productivity apps, chasing the TV networks with YouTube. Etc etc. Today search is only one of the things Google is doing, and it may not be the most important thing.

Today Google's profits come from ads, and that business gives them a reason to keep search weak. They want you to do a lot of searching to find what you're looking for -- and the stuff they find for you for free is competing with the stuff they make money on. So Google actually has a disincentive to make search better.

A few months ago, I used to get into regular debates with Matt Augustine who argued that the fact that companies like Google make so much money from search advertising seems like a bug in the system. Matt would argue that if search engines were really that good at finding what we want, we would never have to click on the results they had been paid to show us unless we were deceived into doing so.

This seems to put a 'glass ceiling' on how good the search engine can be because you don't want people to stop clicking on ads when you make billions of dollars a year from them doing so.

Promoting Google Services at the Expense of the Integrity of Search Results and it's Advertisers

Blake Ross has a blog post entitled Tip: Trust is hard to gain, easy to lose where he writes

But Google lost me today, and it didn’t take much:

Google is now displaying “tips” that point searchers to Google Calendar, Blogger and Picasa for any search phrase that includes “calendar” (e.g. Yahoo calendar), “blog” and “photo sharing,” respectively. This is clearly bad for competitors, and it’s also a bad sign for Google. But I generally support anything that benefits users, including monopolistic packaging. I believe, for instance, that shipping Internet Explorer with Windows was a good move. So why are tips bad for users?
...
The tips are different—and bad for users—because the services they recommend are not the best in their class. If Google wants to make it faster and easier for users to manage events, create a blog or share photos, it could do what it does when you search GOOG: link to the best services. To prevent Google from being the gatekeeper, the company could identify the services algorithmically. But if that sounds familiar, perhaps that’s because Google already works that way. After all, Google is predicated on the idea that the democratic structure of the Web will push the cream to the top. Search for “photo sharing” and you should already get the highest quality services. According to Google, Picasa is not one of them.
...
While advertisers compete to be first in a string of lookalike ads that are often shunted to the side, Google now determines the precise position and appearance of ads tips that are not subject to any of the same rules. Its ads get icons while others don’t, and if you think that’s small potatoes, you are not an advertiser: images boost clickthrough. Google can make a Picasa ad say “Easier to use than Kodak,” but Kodak cannot create an ad that reads “Easier to use than Picasa.” And the kicker: neither the highest quality ads nor the highest quality search results can replace these tips.

The "strategy tax" here is being paid by the search engine and advertising groups at Google. To carry along Google services that Blake points out are not best in class, Google is foregoing ad dollars from a number of lucrative keywords and causing distrust in the search engine by the very power users upon whose backs it rose to fame in the first place. Google used to brag about how unlike other search engines, they don't use misleading ads that people can confuse for search results. However I tend to agree with the last statement in Blake's post

Perhaps the most nefarious aspect of this feature is how it operates within our collective blind spots. Advertisers are happy that Google no longer invades the canonical Ad Results. Technology purists continue to see untainted Search Results. But does my mother make that distinction? How much does a result have to look like a Result to cross the line?
Indeed.

Artificially Promoting it's Products in Search Results

From a comment highlighted in the post Google's Silent Monopoly Redux (Google Responds - Issues Public Statement) which states

But type in "maps". Google is again first. Ahead of Mapquest. Ahead of Yahoo maps. Yahoo also has backlinks out the ying yang. So why is it third? And mapquest has been around forever.. I'm sure there are more links to mapquest.com than to the maps.google.com URL, simply because the millions of web pages that linked their directions to Mapquest from 1996 to 2004 didn't all rush out and change all their links to Google maps in February of 2005 (when it was released), even if Google's is a better product.

Next, try "mail". Despite the fact that Yahoo mail has been around forever, and has all sorts of links, and that Hotmail has also been around forever, Gmail still manages to come up first.

And the most interesting thing about this particular keyword? The word "mail" by itself doesn't even appear on the gmail.google.com page! The words gmail, webmail, and email appear. But not "mail". At least on the Yahoo page, the word "mail" does indeed appear. Yet Google still manages to rank ahead of Yahoo.

Finally, try "answers". Yes, answers.google.com comes up second, rather than first. But answers.yahoo.com comes in third! Is the Google Answers site really getting that many more links than Yahoo's? Especially in light of the fact that Google recently decided to kill it, because almost no one was using it, while Yahoo's usage (and therefore also linkage, no doubt) are skyrocketing?

This claim was actually the most interesting to me since Google is very adamant about the integrity of their search results and claims we don’t accept payment for inclusion in our index, nor do we manipulate search results by hand. I tried a number of these queries myself and was pretty shocked by the results especially when it came to "mail". Here are some screenshots that illustrate the point

1. Search results for "mail" on Google

2. Number of links to gmail.google.com (also the same as gmail.com) according to Google

3. Number of links to mail.yahoo.com according to Google

It's hard to imagine any objective metric that should make Gmail show up ahead of Yahoo! Mail in a search for the word "mail". Of course, this doesn't mean that Google is tampering with search results "by hand". Their algorithm can simply have allowances to rank sites in their domain or linked from their domain higher without having to actually sully their hands by tweaking individual results by hand. Still, if Google is how the world finds information and we are increasingly being pointed to information that financially benefits Google, doesn't that taint the much vaunted claim of the integrity of their search results even if it is being done in an automated manner?


 

December 19, 2006
@ 02:02 PM

Brady Forrest over on the O'Reilly Radar blog just announced that Google Deprecates Their SOAP Search API where he states

In an odd move Google has quietly deprecated their Search SOAP API, will no longer be issuing keys, and have removed the SDK from their site. They did not even issue a blog post about it. They will continue (for how long?) to support existing users, but will not do any bug fixes. They are urging developers to use their AJAX Search API ((Radar post) instead.

The AJAX Search API is great for web applications and users that want to bling their blog, but does not provide the flexibility of the SOAP API. I am surprised that it has not been replaced with a GData API instead. The developer community has been discussing this and do not seem happy with the change. Discussion on the forums have pointed out that Yahoo! has a REST Search API. Live Search also has a SOAP API available.

I find it odd that Brady is surprised by this move. Brady used to work on the MSN Windows Live Search team working on APIs and he should know first hand that the value of Search APIs was always questioned. Unlike data APIs which extend the reach of a service and add value via network effects such as the MetaWeblog API, Flickr API or the del.icio.us API, the search APIs provided by the major search engines do no such thing. With the data APIs one can argue that making it easier for people to add content to sites increases their value, on the other hand making it easier for people to run search queries without seeing highly lucrative search ads doesn't make much business sense.

This reminds me of a quote from Bill Gates taken by Liz Gannes in her report Bill Gates on the Future of Web Apps which is excerpted below

We each got to ask Gates one question. I asked which applications he forecast to live within the browser and which outside of it.

He replied that the distinction would come to be silly from a technical standpoint, but that the necessary movement toward web APIs does present challenges on the business side. “One of the things that’s actually held the industry back on this is, if you have an advertising business model, then you don’t want to expose your capabilities as a web service, because somebody would use that web service without plastering your ad up next to the thing.”

His solution wasn’t very specific: “It’s ideal if you get business models that don’t force someone to say ‘no, we won’t give you that service unless you display something right there on that home page.”

The quote seems particularly relevant now when you consider that Google has replaced a web service with their AJAX Search API which is a widget that is easier to monetize. I'd also note that Scoble telegraphed that this move was coming in his post Google changes its monetization strategy toward a Microsoft one? which implies that Google AdSense will be bundled with usage of Google's search widgets.


 

December 15, 2006
@ 03:09 AM

Moishe Lettvin: Large companies and 'A' talent

But then I got an offer from Google and after a little bit of waffling (I was having much fun with the hackers) I started there back in January. And holy shit I hope I can convey to you what sort of geek heaven I'm in now.

Above I talked about NT4 being the "new hotness" back in '94 -- the guys who made it that way sit right next to me. In the same office. And that sort of expertise is everywhere here... it seems like every office is occupied by at least a couple of industry leaders, guys whose names you'd recognize if you're even a casual observer of geek culture.

Google's culture values independence and transparency of communication in ways I didn't think were possible at a large company. We've of course got our 20% time, but beyond that there's a sense that everyone here is competent enough and trustworthy enough to be clued in to many parts of the business -- not just engineering -- which would typically be hidden. That trust nets huge gains in loyalty and excitement.

There aren't many places in the world where you could can come up with the idea for a feature or product, implement it, and launch it to an audience of millions, with the infrastructure to support it. Yes, you can do it at a startup or on your own, but getting eyeballs and servers is non-trivial. For every YouTube there are hundreds of sites nobody's heard of.

Aaron Swartz: The Goog Life: how Google keeps employees by treating them like kids

The dinosaurs and spaceships certainly fit in with the infantilizing theme, as does the hot tub-sized ball pit that Googlers can jump into and throw ball fights. Everyone I know who works there either acts childish (the army of programmers), enthusiastically adolescent (their managers and overseers), or else is deeply cynical (the hot-shot programmers). But as much as they may want to leave Google, the infantilizing tactics have worked: they're afraid they wouldn't be able to survive anywhere else.

Google hires programmers straight out of college and tempts them with all the benefits of college life. Indeed, as the hiring brochures stress, the place was explicitly modeled upon college. At one point, I wondered why Google didn't just go all the way and build their own dormitories. After all, weren't the late-night dorm-room conversations with others who were smart like you one of the best parts of college life? But as the gleam wears off the Google, I can see why it's no place anyone would want to hang around for that long. Even the suburban desert of Mountain View is better.

Google's famed secrecy doesn't really do a very good job of keeping information from competitors. Those who are truly curious can pick up enough leaks and read enough articles to figure out how mostly everything works. But what it does do is create an aura of impossibility around the place. People read the airbrushed versions of Google technologies in talks and academic papers and think that Google has some amazingly large computer lab with amazingly powerful technology. But hang around a Googler long enough and you'll hear them complain about the unreliability of GFS and how they don't really have enough computers to keep up with the load.

"It's always frightening when you see how the sausage actually gets made," explains a product manager. And that's exactly what the secrecy is supposed to prevent. The rest of the world sees Google as this impenetrable edifice with all the mysteries of the world inside ("I hear once you've worked there for 256 days they teach you the secret levitation," explains xkcd) while the select few inside the walls know the truth -- there is no there there -- and are bound together by this burden.

The truth is always somewhere in between.


 

Keith Teare of Edgeio has a blog post entitled De-portalization and Internet revenues where he writes

7. Publisher driven revenue models will increasingly replace middlemen. There will be no successful advertiser driven models in the foothills, only publisher centric models. Successful platform vendors will put the publisher at the center of the world in a sellers market for eyeballs. There will be more publishers able to make $180,000 a month.
8. Portals will need to evolve into platform companies in order to participate in a huge growth of Internet revenues. Service to publishers will be a huge part of this. Otherwise they will end up like Infospace, or maybe Infoseek. Relics of the past.
9. Search however will become more important as content becomes more distributed. Yet it will command less and less a proportion of the growing Internet traffic.
10. Smart companies will (a) help content find traffic by enabling its distribution. (b) help users find content that is widely dispersed by providing great search. (c) help the publishers in the rising foothills maximize the value of their publications.

I find Keith's post interesting especially when juxtaposed against Fred Wilson's take on how the big Web companies like Yahoo! can relate to this trend in his blog post The De-Portalization of the Internet (aka What I Would Do If I Were Running Yahoo!) where he writes

Today, we shop directly with the Internet merchants we like or we use a shopping search engine to find what we want. We can look for jobs on Indeed, meet people on MySpace or Facebook, find roomates on Craigslist, and use Meebo for instant messaging. It's rarely true that the best of breed service exists on a "portal". The portals continue to buy best of breed services like Flickr, but now they let the service continue to exist on the web with its own look and feel and URL structure.
...
So if you buy that the web has been de-portalized, what do you do if you run the largest portal in the world? I think its pretty simple actually. Yahoo! needs to offer its users and customers (advertisers) the ability to get the same experience they get on Yahoo! all over the web. They need to stop thinking about keeping their audience on Yahoo.com and start thinking about serving their audience wherever they are on the web. They need to stop thinking about selling ads on Yahoo.com and start thinking about selling ads all over the web.
...
So what are some concrete things they need to do? Well first, they need to improve their search service. On a de-portalized web, it all starts with search. I never hear of companies that have 80 percent of their traffic coming from Yahoo! I hear of companies all the time that have 80 percent of their traffic coming from Google. Yahoo! may have 28% of all Internet searches, but for some reason that I am not sure I completely understand, Yahoo! does not generate 28% of Internet traffic.
...
And Yahoo! needs to get its YPN (Yahoo! Publisher Network) service in gear. They need to offer advertisers the ability to reach people when they are not on Yahoo! They've done some things recently, like the eBay partnership, that suggest they are headed in that direction. But I would urge them to move faster in this direction than they are moving now. It might mean buying some ad networks instead of just investing in them.

This is probably the best advice I've seen on this topic and one I'm sure a lot of folks over here at MSN Windows Live would nod their heads in agreement as they read Fred's advice. The one thing missing from Fred's advice is how exactly Yahoo! should "offers its users and customers (advertisers) the ability to get the same experience they get on Yahoo! all over the web". I'm not sure Fred realizes it but Yahoo! is already halfway there if you look at a number of their initiatives. For one, there are the numerous APIs for Yahoo! services which enable websites and Yahoo! users to incorporate Yahoo! content and services wherever they want on the Web. More importantly, there is now Yahoo! Browser Based Authentication (BBAuth) which is a low cost way for any site on the Web to appear to end users as a member of the Y! network of services since it accepts Yahoo! credentials. Yahoo! is making a lot of the right moves, their big problem now seems to be whether they can evangelize market these initiatives to their customers and other websites in a way that increases adoption. Ideally, they need to show websites how to make that $$$ by partnering with Yahoo!, Google has the advantage in that they have lead with providing $$$ to websites outside their network and now have in that is difficult to beat when it comes to "giving users the Google experience wherever they are on the Web". One could argue that Google Custom Search Engine is a good example of Google embracing the de-portalization trendin the only Google service that end users actually care about.

When it comes to the importance of search, one thing to note is how delicate of a position the major commercial sites such as Amazon and eBay are in. The pattern with the major portals search engines is that they look for what customers are searching a lot for and then provide that as a service. Witness Google's integration of Google Video into the main search page when they realized how much traffic they were sending to YouTube. However the YouTube brand was too strong to be defeated by such tactics and eventually Google purchased the site instead of competing with it. Thus far, Google has embraced de-portalization by providing ads for commercial sites like Amazon but what happens when they realize that they send a ton of traffic to the Amazon and could be getting a cut of the referral fees? I'd keep an eye on Google Checkout if I worked at Amazon or eBay. I suspect that it is just a matter of time before paying the Google tax will be part of the cost of doing business on the Web, in the same way giving Google a cut of your advertising revenues (i.e. being a Google AdSense customer) is almost a given when venturing into the content business on the Web today.

Embracing de-portalization means becoming the ultimate middle man. I remember when erecting Internet Toll Booths was a bad thing. ;) 


 

December 6, 2006
@ 02:50 AM

Niall Kennedy has been on a roll in the past couple of weeks. He has a blog post entitled Brands will be widgetized, but who is the author? which tackles the interesting problem of widgets, branding and customer confusion. He writes

Sites with personal user data placed behind a username and password may be subject to new types of phishing attacks from the widget web. A user will likely locate your service's widget through the widget provider's directory, searching for terms such as "Gmail" and "eBay" to access their latest mail messages or watched auction items. These widgets will prompt the user for their login information before delivering personalized information from each service, leaving the trust of a brand in the hands of a third-party developer who may or may not act in the best interest of the data provider.

If Google Mail and eBay worked directly with the large widget producers to establish certified or trusted widget status they could reduce opportunities available for third party widgets offering enticing functionality to send messages to a remote server with collected user data. The trusted, certified, or verified seals provided by each widget platform is one way to ensure users receive the official product and not a knock-off.

This issue has been rattling around in my head ever since I wrote a Flickr gadget and a Blufr gadget for Windows Live Spaces. After all, I don't work for either company yet here I am writing gadgets that are being used by hundreds of users in their name. Who ends up getting the taint if my gadget is buggy or causes some problems for the user? Me or Flickr? What happens if legitimate looking gadgets like mine are actually fronts for phishing attacks? How can Flickr protect their users and their brand from malicious or just plain sloppy developers? I like the idea of the major widget galleries like Windows Live Gallery, Yahoo! Widget Gallery and Spring Widgets coming up with a notion of trusted or certified gadgets but it seems like an unfortunate hoop that web sites now need to jump through to police their brands on the various widgets sites on the Web.  Reminds me of trademark holders having to rush to register their brand name as a domain whenever new TLDs are opened up.

PS: This is one of the reasons you don't see a bunch of Windows Live gadgets out there today. The brand dilution and phishing problem is a real one that worries lots of folks over here.


 

First it was Yahoo! Mail that swallowed the AJAX pill only to become unusably slow and now it looks like Yahoo! TV is another casualty of this annoying trend.

Dave Winer writes

Yahoo says they improved Yahoo TV, but imho, they broke it. The listings page, which until today was the only page I knew or cared about (they just added a bunch of community features) took a few seconds to load, now it's an Ajax thing, and it loads as you scroll. Great. There's a delay every time I hit Page Down. Now instead of finding out if there's anything on in seconds it takes minutes. That's an improvement? 

In his post entitled Yahoo TV Goes 2.0. Argh.Paul Kedrosky writes

Well, Yahoo in its wisdom has launched a 2.0-ified version of its TV listings tonight, complete with an Ajax-y interface, cool blue colors, social rating of programs, etc. That's all swell, and frankly I wouldn't care one way or the other (other than they broke my URL for full listings), but the darn thing is sooooo much slower than the old listings. Tables have to get populated, drop-downs have to ... drop, and sliders have to slide while data creakily loads.

It's really irritating -- so irritating, in fact, that rather then wade back in to find out what time tonight the new Frontline episode is out about credit cards, I think I'll just watch it on the Frontline site.

Seriously, who's making these decisions at Yahoo? Don't they realize that slower websites cost them money regardless of how buzzword compliant it now makes them?


 

Erik Selberg, a developer on the Windows Live Search team, has a blog post entitled General disarray at The Big 3 where he writes

given the recent trends in query share. I’ll summarize for those who don’t want to read to the bottom of Danny’s post:
sullivan-ms-query-share-dorp.jpg
Greg’s take:

Ouchie. As Danny says, “[Not] a pretty picture for Microsoft … They haven’t held share. It’s drop, drop, drop.”

It really is remarkable how badly Microsoft is doing against Google. I never would have thought that, nearly four years after they started their “Underdog” project to build a Google-killer, Microsoft would not only be badly behind in search, but also actually losing market share.

Well, what did anyone really expect?

Let’s put some things into context. First, all of the above is brutally, painfully true. Google hired smart, self-starters who are into big risk / big reward.
...
Yahoo is just in a rough place. They’ve got Google dominating, and they’ve got us coming up from behind.
...
And then there’s us at Microsoft bringing up the rear with declining query share. Well… yeah. While our management set the goal of having relevance that beat Google after 2 years (then 3, and I believe 4 now…) it’s not realistic to think that it can be done quickly. If you ask Google, Yahoo, or the fine SEOs at WebMasterWorld or other such places, they’ll all say that Live Search has increased in quality over the years so that it’s much closer to Yahoo and Google. Not yet better, but no longer laughable. And yeah, we’ve done our own share of copying feature parity, and we’re starting to do a few things that cause Google and Yahoo to do the same (ok, noODP is a small feature, but it’s a start!).

Here’s the honest truth… Microsoft will continue to lose share until it can make Live.com something people chose versus just the IE default. That will happen when the average person starts to see Live.com as a bit better than Google. Right now, Google wins on brand (people like them a lot) and quality, so it’s to be expected that existing Yahoo / Live customers will migrate to Google than vice-versa and new customers will pick Google more than Live or Yahoo. Google is making people focus on features, which should tell people that they’re worried about how we’re catching up, and are going to put more people on their core products to keep and extend their lead. So it’s going to be a tough, tough battle for Microsoft to get there…

As I read Erik's post, one phrase kept repeating itself in my head over and over again; "Stay the Course...Stay the Course...Stay the Course". I find it amazing that people like Erik still think that competing with Google is about being a bit better than their search engine or having relevance that beats theirs in a few years. Competing with Google's search engine is no longer about search results quality, it is about brand and distribution. This is why even though the search engine that powers MSN Search or Windows Live Search has gotten better and better Microsoft's share of search engine market has dropped almost 50% since it announced that it would launch its own search engine to compete with Google's. Competition with Google really should focus on addressing both of these points.

Brand

The verb 'google' is now in the Mirriam Webster dictionary. That is the power of brand. Anyone who regularly uses the Internet be they young or old thinks Google is synonymous with search.

Anecdote: My girlfriend once told her kids we were takin them to the zoo and her seven year old jumped on computer and went straight to http://www.google.com to fiind out what animals she'd see that day. 

Distribution

The combination of the proliferation of search toolbars and a new generation of Web browsers with built-in search boxes (e.g. IE 7 and Firefox) have reduced the need for users to actually go to websites to perform a search. This means that it is now very important to be the search engine that is used when a user enters a search directly from their browser. Guess which search engine is the one used by your browser if you
  1. Are you a user of the Firefox browser?
  2. Are you a user of the Opera browser?
  3. Are you a user of IE 7 and have installed Adobe Acrobat?
  4. Are you a user of IE 7 and have installed the Java runtime?
  5. Are you a user of IE 7 and have installed the WinZip archive utility?
  6. Are you using a newly purchased Dell computer?
  7. Are you a user of the Google Toolbar?
Yes, the answer is Google in every case. So even if you are an Internet n00b who hasn't made up their mind about which search engine to choose, there is a large chance that the default search engine you end up using thanks to recent innovations in IE 7 and Firefox will be Google.
 

November 22, 2006
@ 07:02 PM

Nick Carr has a blog post entitled Eric Schmidt's tough talk where he writes

Google CEO Eric Schmidt has been coy in discussing his company's ambition to create an online alternative to Microsoft Office. Just a few days ago, at the Web 2.0 Summit, Schmidt "played the semantic game" in discussing office suites, reported Dan Farber. Schmidt claimed "that Google is developing applications for just 'casual' use. 'We don’t call it an office suite. It's not targeted at the [Microsoft] Office – we never made that claim.'"

But a very different, and much more aggressive, Eric Schmidt appears in the Economist's new "World in 2007" issue. Schmidt contributes an article titled "Don't bet against the Internet," in which he makes a striking prediction. Next year, he writes, "we’ll witness the increasing dominance of open internet standards." These standards "will sweep aside the proprietary protocols promoted by individual companies striving for technical monopoly. Today’s desktop software will be overtaken by internet-based services that enable users to choose the document formats, search tools and editing capability that best suit their needs."

It's refreshing to see Google stop playing coy and be straightforward about their ambitions. At the Web 2.0 conference last year, Sergey Brin was coy about their plans when questioned by John Battelle. Given Google's significant market valuation they need to be making a lot more money than they are doing now to satisfy the markets. What better than targeting a multi-billion dollar business of a fierce competitor which is ripe for disruption? 

Now that their ambitions have been laid bare, I really hope this changes Microsoft's Office Live strategy. A lot  of  people expected Office Live to be a hosted version of Microsoft's Office suite. It is clear there is a pent up demand to bring office applications in the Web era, however it is unclear whether the simplistic division of desktop versus web applications is the right way to view this evolution. I believe the truth is that there is a continouom in which these applications should live and some applications sit better on the desktop end (e.g. word processing) while others sit better on the Web end (e.g. email reading). Ray Ozzie has said similar things in his speech at a recent Microsoft Financial Analyst Meeting.

First a revamped UI for Microsoft Office and now Google jumping into the Web Office game with both feet? 2007 is going to be an interesting year for Office productivity software.


 

Jeremy Zawodny has a blog post entitled Yahoo's Peanut Butter APIs where he writes

Over the weekend someone sent me email that, among other things, said roughly: "I’m glad to see that APIs weren't on the list of things Brad Garlinghouse wants to get rid of. That means you're safe, right?"
...
Now, if you're one of the people who's asked me what I think about all this... here's my answer: Brad is very right about some things and terribly wrong about others.

A couple of us were chatting about the memo at lunch yesterday and I realized that if I was a Yahoo! employee my spider sense would be tingling like mad. I'd also be considering talking to my peeps at GOOG and MSFT to see if they had any openings I was interested in just to hedge my bets. There are the three things about the memo that made me come to this conclusion

  1. The memo recommends 15% - 20% head count reductions. This means that Yahoo! executives are contemplating firing one in five Yahoo! employees. How is it going to be decided whose job is safe and whose isn't? Layoffs are a demoralizing affair and often don't eliminate the right people especially since the really smart people know to desert a sinking ship instead of hanging around to see if they draw the short straw.

  2. A supposed senior VP at the company seems unable to tell the difference between the audience for Flickr vs. Yahoo! Photos or the difference between Konfabulator widgets and plugins for the Yahoo! IM client.Out of touch executives tend the bundle similar products in their mind and view it as redundancy without understanding the context in which they exist. This often leads to merging of products and the pissing off of customers. If Yahoo! actually goes through with the implied recommendations from this memo expect Netscape.com-like traffic drop offs by some of their users.

  3. People in glass houses shouldn't throw stones. If you look at Brad Garlinghouse's record it sounds like he is actually an example of the problem he rails about in his memo as opposed to the savior people in the blogosphere are calling him for penning the memo. See the article Yahoo's Misguided Manifesto which contains the following excerpt

    Here again, Yahoo! should start with accounting for Garlinghouse's performance. Under his watch, Yahoo! Messenger let a huge opportunity for voice-over-Internet protocol, or VoIP, slip through its fingers as eBay ( snapped up Skype. And Yahoo Mail dropped behind Google's Gmail as the most prestigious Web-based email domain.

    That performance has been more chunky than smooth, yet Yahoo! has gotten off easy. Earlier, Garlinghouse was CEO of VoIP leader Dialpad, which promptly spiraled into bankruptcy, but not before Garlinghouse laid off 90 of his 140 employees. A 2002 case study of Dialpad in the Harvard Business Review discussed how Garlinghouse struggled with a failed business model while rival Net2Phone won a $1.4 billion investment from AT&T as well as deals with Microsoft and Yahoo!.

    That's right, didn't Garlinghouse's group purchase OddPost 2.5 years ago and yet still hasn't figured out how to integrate the offering into Yahoo! Mail besides making a molasses slow Web-mail experience that is still in beta? Is this the guy whose going to save Yahoo! and show it how to integrate Flickr into Yahoo! Photos or del.icio.us into Yahoo! MyWeb. Good luck, you're going to need it.


 

November 20, 2006
@ 05:51 PM

EXHIBIT A

EXHIBIT B


 

If you are a 'Web 2.0' watcher by now you've seen the hubbub over the Peanut Butter Manifesto memo which is an Yahoo! internal memo authored by Brad Garlinghouse, a senior VP at the company. The memo is a rant against the typical list of woes that face big companies (e.g. the contradiction of being spread too thin yet having too many people, duplicative products and misaligned goals across the company). What I've found most interesting hasn't been the memo but instead the responses to it.

For example, in a blog post entitled Yahoo’s Brad Garlinghouse Makes His Power Move Mike Arrington views the memo as a clever attempt at an internal coup by Brad Garlinghouse. However even more interesting is the following comment in response to Arrington's post by someone named gullova which is excerpted below

Yahoo continues to get whipped by Google because its leaders can not get the product and engineering teams to focus on the right projects.

Witness Panama (the new ad system). Yahoo has been talking about Panama since early 2004. Yet the product they are launching is barely what Google had 2 years ago.

They threw hundreds of people at Panama, hurting other projects along the way, yet ultimately they are building the wrong product. Panama is far too focused he needs of search advertisers, which makes little sense since Yahoo’s search share has been shrinking since the day they dropped Google and launched their own search engine.

Had Panama instead been about display advertising, Yahoo could have at minimum increased monetization on Yahoo, which lets remind ourselves is still the largest site on the web and which they could monetize at 0% TAC (so its all gravy to the bottom line).

Yahoo is full of guys like Brad who can articulate themselves well and give great presentations. The problem is that the engineering team doesn’t listen to them, and the executive team doesn’t make them listen.

If they really want to get listened to, they should just shut down Panama and run Google ads instead. Its not a stretch to say they’d probably make more money.

The last sentence is the kicker for me. What if instead of competing with Google by funding its own search engine and advertising product, Microsoft partnered with Google like AOL has done? One of the pros of doing this are that it would free up a huge commitment of resources in competing with an industry leader that is years ahead of Microsoft to then focus on building applications that grow its audience directly which is then left to Google to monetize. Another possible pro is that the average revenue per user (ARPU) may go up with Google AdSense + AdWords being used to monetize Windows Live and MSN audiences as opposed to Microsoft's offerings. However a couple of minutes searching online doesn't given enough public data to determine whether this would be the case or not.

The cons are many. The first is that Microsoft would be seen to be admitting defeat if it switched to using Google's monetization engine although from a purely business perspective this isn't a significant con. Another con is that Microsoft would be enriching a competitor who is targetting one of its cash cows for obsolesence. See Google Docs & Spreadsheets, the JotSpot purchase and Google Apps for your Domain which are all attempts at attacking the success of Microsoft Office and related products like Microsoft Exchange. In this case, Microsoft would be guilty of being penny wise and pound foolish. The final con and perhaps the biggest problem with Microsoft going with the Google monetization engine is that it makes Microsoft entirely dependent on a single customer/supplier [who was also a rival] for a majority of the revenue from its online businesses.

When I started this post I tried to keep an open mind about the idea but by the time I finished writing it was clear that this is a bad idea. Funny how that happens.


 

In recent times whenever people compare the quality of search engines they usually focus on the 'relevance' of the search engine results. Over at MSN Windows Live we've taken this seriously and there have been numerous reports over the past year or two about how much our search engine relevance has been increasing. However I've recently been wondering whether 'relevance' is really all that relevant today and there are other factors that I consider more important than whether the most relevant web pages are returned for my web search. Below are examples from Google and Live Search to illustrate my point

  1. Search for "Marvel Ultimate Alliance" on Google

    Note the options to 'Refine Search results' that gives you links to queries for screenshots, cheats and reviews along with finding the most relevant web site that matches the query.

  2. Search for G-Unit in Live Search

    Note the 'Related Searches', list of Top Downloads for the rap group and a link to the G-Unit page on Rhapsody along with finding the most relevant web site that matches the query.

  3. Search for "skate king bellevue" on Yahoo!, Google and Live Search, all of which not only bring up the most relevant website that matches the query but also the business's phone number and address on a map as well.

What I'm getting at is that relevant search results is on the way to being a commodity. Yahoo! search, Google search and Live search all give me pretty relevant search results most of the time. We are at the stage in the world of web search where what will keep a search engine on top or make it rise to prominence if it isn't on top is how much more it does beyond just finding relevant web pages.

This isn't a startling revelation and I'm sure all the folks working at the major search engines already realize this but it was a new insight to me. :)


 

October 30, 2006
@ 03:35 PM

I read an interesting pair of posts about Web startups competing with big companies like Google and Microsoft over the weekend. The first was a post by Bill Burnham entitled Search Applications: Search Startups Are Dead, Long Live Search Startups where he writes 

In response to a question about the prospects for the myriad of search start-ups looking for funding Peter basically said, and I am paraphrasing somewhat, that search start-ups, in the vein of Google, Yahoo Ask, etc. are dead.  Not because search isn’t a great place to be or because they can’t create innovative technologies, but because the investment required to build and operate an Internet-scale, high performance crawling, indexing, and query serving farm were now so great that only the largest Internet companies had a chance of competing.

Priced Out of the Market
While the comment might strike some as self-serving, the fact of the matter is that it is true.  Any start-up trying to displace Google, Yahoo, or even MSN or Ask (or for that matter any VC trying to fund them) should just get in their car (or hop on a plane) and go look at Google’s new server farms at The Dales in Oregon.  And if that doesn’t convince them they should head up the Columbia river a bit and check out Microsoft and Yahoo’s digs.   The costs to compete in core search , are now simply to high.

Bill then goes on to argue that the opportunity may lay in building applications on top of the APIs provided by the big software companies such as the Alexa Web Search Platform instead of trying to compete head to head with companies that have already invested hundreds of millions of dollars in building out their search infrastructure. In a post in response to Bill Burnham, Tim O'Reilly agrees that "we're entering the platform phase of Web 2.0, in which first generation applications are going to turn into platforms".

Dave Winer comes to the same realization but with a different conclusion in his post Someday search will be old too where he writes

Many years ago, when the Internet was still the domain of geeks, researchers and college students, the smart folks often said that the opportunities for new software companies were over, it simply required too much scale to compete in an industry dominated by Lotus, Microsoft and Ashton-Tate. Now it's clear how ridiculous that was, even though it was correct. The next layer comes on not by building on the old layer (a trick, the guy you're building on will eat your lunch), or re-doing what they did (what the naysayers correctly say you can't do), but by starting from a different place and building something new, and so different that the old guys don't understand it and don't feel threatened by it.

Dave seems to be disagreeing with Bill Burnham and Tim O'Reilly that building on the platforms provided by the big software companies is a good idea because the companies can turn around and compete with you. Although I can see Dave's point, simply having the same base platform doesn't mean you can build the same application. We all have access to Linux, Apache, MySQL and PHP but how many people have or can build something like Flickr? You should definitely examine the risks when building on any platform but eventually you have to be ready to face off against a competitor that may have access to the same or a similar platform as you. You're development platform cannot be your only differentiator.

I do agree more with the spirit of what Dave Winer is recommending than what Tim O'Reilly  & Bill Burnham are. If I ever founded a startup, it would take advantage of the knowledge I have of what big companies like Microsoft are good at and what they aren't. For example, YouTube would never have come out of Microsoft because the company would have been too scared of lawsuits. Of course, now that YouTube has money they are getting hit up from every angle so that fear does make sense for a big company but not a small startup especially if the exit strategy is to flip it


 

In his blog post entitled Yahoo Bookmarks Enters 21st Century Mike Arrington writes

Yahoo is unveiling an entirely new Bookmarks product this evening at new.bookmarks.yahoo.com - new interface, new back-end, the works. A screencast created by Yahoo developer Tom Chi is here which gives an excellent overview of the service (Chi also created the background music for the screencast). Compare that to the existing Bookmarks product (screenshot is here) and it’s clear how significant the overhaul is.

Yahoo Bookmarks, while invisible to most cutting edge web users, still claims around 20 million active users (compared to only 1 million for del.icio.us).

Until today, Yahoo Bookmarks (which is a separate product from del.icio.us and My Web) stored only the URL, title and comment for a particular bookmark. The new product caches all text on the page, stores a thumbnail view, and allows both categorization (folders) and tagging of each bookmark.

There are two things that I found striking about this announcement. The first is that Yahoo! has three bookmarking products; del.icio.us, My Web and Yahoo! Bookmarks. The second is that it looks like my shunning del.icio.us while continuing to use Yahoo! Bookmarks isn't so weird after all, there are twenty times more people using Yahoo's regular bookmarking service compared to its social bookmarking product.

At this rate it looks like del.icio.us is destined to be a niche service unless something radical is done such as merging Yahoo! Bookmarks and del.icio.us into a single service. Of course, given the user outcry when Netscape.com revamped to be more Digg-like, the folks at Yahoo! may not have the stomach for this.

Faint heart never won fair lady. 


 

In the past few months there have been a couple of announcements from the big search engines such as Yahoo! and Live Search on the topic of enabling people to build their own custom search engines. Google has finally showed up at the party with their own offering which was unveiled today. Below are my thoughts on their offering versus that of Windows Live.

Google Co-op

In his blog post entitled Review: Custom Search Engine Matt Cutts of Google writes

Google just announced something that I’m really jazzed about: Google Custom Search Engine. Several people mentioned that Google’s Accessible Search was built by using Google Co-op under the hood. Co-op has opened much of that power up to the public, so that anyone can build a custom search engine.

Most custom search engines (whether it be Google’s free sitesearch or Yahoo! Search Builder) only let you select one site to search, or you can offer websearch. Even Rollyo only lets you search over 25 sites.

This new offering lets you easily add hundreds (thousands?) of urls. You can search over ONLY the sites you choose, or (my favorite) you can apply a boost to the sites you choose, with regular websearch as a backfill. That’s really nice, because if your chosen urls talk about a subject, you’ll often get matches from those urls, but if the user types something completely unrelated, you’ll still get web results back. So it’s a true custom search engine, not just an engine restricted to showing matches from some domains.

You can also choose to exclude results from different sites. As far as I can tell, this happens in pretty close to real-time, even for complex url patterns. For example, I added the pattern “google.com/*” and started to get results from the Google directory, so I excluded “google.com/Top/*” and the Google directory results went away immediately.

There is also a screenshot included in Mike Arrington's post at TechCrunch entitled Google Co-op Launches which is excerpted below
This isn’t new - Rollyo, Eurekster and Yahoo already have similar products. But Google is also offering, as an option, to bundle the service with Google Adsense ads and share revenue with websites that embed the custom search engine into their site. Only Eurekster currently shares revenue with users. Yahoo’s product, which got a lot of press at launch, has barely been mentioned in the nearly three months since then.
I didn't even realize that Yahoo! had an offering in this space until reading the TechCrunch entry. This doesn't seem to have gotten that much blogosphere love.

Live Search Macros

The Windows Live Search team wrote about the changes to the Search Macros feature originally announced in March in their blog post entitled Create your own search engine (an update to Live Search Macros) which states

Search Macros are personalized search engines for any topic area of interest.  You can create them, use them, share them with friends or discover macros created by the community on Windows Live Gallery.

I’d like to use this post to give you a basic overview of using and creating macros.  We’ll use future posts to dive into more of the nitty gritty on specific macros features.

Finding and using macros

Users of the first Macros release told us that using a macro was difficult and not very user friendly.  In this release, every macro now has its own homepage and human readable URL.  This makes them much easier to use, bookmark, and send to friends over email or IM.  For example, check out the homepage for the Reference Sites Search Engine macro (at http://search.live.com/macros/livesearch/reference):
...
Enter a search term on this page and press Enter. You’ll be taken to the main Live Search page to see your results.

...
On the results page you'll see that the macro’s name appears in the search bar at the top of the screen.  This enables you to switch back and forth between Web, Images, Local, QnA and your favorite macros.

Here are some macros to try:

You can also find many more in the Windows Live Gallery!

Andy Edmonds from the Live Search team has written a couple of blog posts about the cool things you can do with search macros such as Search Macros Recap: DiggRank and the Power of Trusted Networks
Macros are often compared to Rollyo or other site bundling search offerings, but I hope the blog post describing LinkFromDomain, LinkDomain, and featuring other operators, sets the record straight.  Defining a set of sites to search is cool, and an idea well due to be commonly available. To be fair, Rollyo's UI and integration is slick, but  Micah Alpern hacked up a search of his blog, the blogs he linked to, or the web at large with a Google API hack back in 2003!

Using the link domain operators, you can go well beyond a simple set of sites.  You can:

  • keep a living list of the sites that link to you and search them
  • keep a living list of the sites you link to and search them
  • do the same for a set of trusted sites

Access to other advanced syntax differentiates further from simple site search amalgamations.  Heck, Scoble pontificated about a search engine that excluded blogs that participate in pay per post.  While I didn't figure out a way to focus this on only those PPP bloggers who don't disclose their interest, I think it's impressive that the basics can be done at all.  It's called macro:andyed.realBloggers, and uses -inbody:counttrackula.com to exclude sites that use the PPP tracking script (I think!) and hasfeed: to restrict to blogs (or other pages with syndication).

Super Hubs: DiggRank
The promise of personal networks of trust in information retrieval is not fully realized by the macros offering, but it's an important step in the right direction.  For super-hubs, like Digg or Delicious, linkFromDomain captures some really interesting human attentional residue.

Let me introduce macro:andyed.DiggRank. Try it for:

The Bottom Line

I tried out both services as well as Yahoo! Search Builder and they all seem to have some room for improvement. Both Google & Yahoo! have primarily built a way to add a custom search box for your site. Windows Live Search is primarily about adding your own customized search results to your search engine of choice. I think both scenarios should be covered by all the services. I think Live Search should give me the option of adding a search box on my blog that is powered by a search macro I wrote. Similarly, I'd like to be able to perform custom searches from the Google or Yahoo! search UI without having to remember how to get to http://google.com/coop/cse/ or http://builder.search.yahoo.com/. I have to agree with Sergey Brin here, Features, Not Products. This is yet another Google service that I have to perform a Google search for before I can find it and use it (others are Google Music Search and Google Blog Search).

One thing I do like about Google and Yahoo!'s options is that they provide a more user friendly UI for creating complex searches than Live Search which provides you with direct access to the search operators. This is more powerful and desirable to geeks like me but it is not very user friendly for the non-geek. A checkbox with 'prefer search results from these sites' is preferable to crafting a search query with "prefer:http://www.25hoursaday.com AND prefer:http://www.rssbandit.org".

The management page for Google Co-op needs a lot of work. It is sparse in the typical Google way but it also doesn't seem coherent nor does it give you enough information about what you can or should be doing. The management page for Yahoo! Search Builder is a lot more coherently organized and aesthetically pleasing. The search macro management page for Live Search also could do with some improvement, primarily in simplifying the process for creating complex macros.

PS: Revenue sharing is a nice touch by Google and I'd be quite surprised if Yahoo! doesn't follow suite soon.


 

By now I'm sure you've gotten your fill of the news that Google is purchasing YouTube. The most interesting reactions I've seen are the the video of the YouTube founders talking about their newfound wealth on YouTube and the BusinessWeek interview with Steve Ballmer. After thinking about it for a few days, I've come to suspect that this may be an example of the Winner's Curse especially when you consider that the $1.6 billion Google is paying isn't all they have to pay. They'll also have to make deals with the major media content producers (i.e. movie & television studios, record labels, etc) which they've already started doing. I wouldn't be surprised if this deal eventually ended up costing north of $2 billion when all is said and done

Unfamiliar with the Winner's Curse? See the article Nasty Auctions which contains the following excerpt

Imagine a professor who holds up a $20 bill in a class and auctions if off to the high bidder. What do we expect to happen? It would make no sense for anyone to anyone to bid higher than $20, and if the high bid is below $20, someone in the class will have an incentive to bid a bit higher. The end result is that the $20 bill should fetch a high bid of something very close to $20, if not $20 itself. This image of people bidding up the value of an item to its true value is used repeatedly when economists discuss markets. However, auctions are not always so nice.

Imagine instead that the professor presents his class with a bottle of coins. He lets them inspect the bottle, but they cannot open it and it is impossible to count the money in the bottle. He then asks everyone to write down a bid, and accepts the high bid. When this experiment is run in actual classrooms, the end result is that the average bid is less than the value of the bottle, because people are risk-adverse and will bid less than they what the expect the value to be. Some people err in underestimating the value of the coins in the bottle, but others err in overestimating their value. Usually one of the people who overestimate the value ends up winning the bid, and it is very common for the bid to be higher than the true value. Economists have dubbed this phenomenon "the winner's curse," and find it a featue of bidding for oil-exploration rights and for free-agent baseball players
.

Only time will tell whether this deal turns out to be a bargain like Rupert Murdoch purchasing MySpace or a bad move like the AOL<->Time Warner merger. One thing's for sure, it probably won't be considered as wacky as the eBay<->Skype purchase which will probably never make sense.


 

Via Jeremy Zawodny I noticed that Yahoo! has finally launched their Browser Based Authentication (BBAuth) system which they announced at ETech earlier this year. What does BBAuth do?

To use BBAuth, you'll need to do the following:

  1. Register your application

    First you need to register your application with Yahoo!. The process requires that you describe what your application does, provide contact information, set your application's endpoint URL, and select the Yahoo! services to which your application needs access. Some services may divide their API calls into subsets, or scopes. For example, a service might group its read-only methods into a single scope.

    When you complete registration, Yahoo! provides you with an application ID and shared secret for making authenticated service calls.

  2. Log in your users

    Your application cannot access a user's personal data until the user grants your application limited access to their data. To do this you must direct your users to a specialized Yahoo! login page. Once the user enters their Yahoo! user ID and password, Yahoo! displays a Terms of Service page and lists the data which your application may access. If the user grants your application access, Yahoo! redirects the user to your site. The redirect URL contains a token that you use to retrieve the user's credentials.

  3. Use the user's credentials to make web service calls

    Now that you have the user's token, you can use it to retrieve an auth cookie and a WSSID, which together represent the user's credentials. The user's credentials last for one hour, and you must supply them for each authenticated web service call.

This is very similar to Google Account Authentication Proxy for Web-Based Applications. However Yahoo! doesn't seem to have a good story for desktop applications that want to use their APIs on behalf of a user (e.g. a desktop photo management application that wants to upload photos to a users Yahoo! Photos account). On the other hand, Google's authentication system for developers actually does cover the desktop case with Account Authentication for Installed Applications which even goes as far as incorporating CAPTCHAs which the desktop app needs to show to the user as they log them in. The only problem is that unlike the Web case, the desktop application actually collects the username and password which I've already mentioned is a big no-no. However the alternatives have trade offs which I can't blame the Google folks for rejecting. I still can't come up with a solution to this problem that I am 100% comfortable with.

Props to the folks at Google and Yahoo! for opening up their systems in this way. One thing I definitely don't like is that both Google via Google Account Authentication and Yahoo! va BBAuth have shipping code that allows developers to authenticate users that use their services while at Microsoft we're still just talking about it. We need to up our game.


 

September 28, 2006
@ 06:05 PM
Steve Yegge, who works at Google, has a blog post entitled Good Agile, Bad Agile which has a lot of really interesting bits. The first is his take on the origin of Extreme Programming where he writes
So some of the consultants began to think: "Hey, if these companies insist on acting like infants, then we should treat them like infants!" And so they did. When a company said "we want features A through Z", the consultants would get these big index cards and write "A" on the first one, "B" on the second one, etc., along with time estimates, and then post them on their wall. Then when the customer wanted to add something, the consultant could point at the wall and say: "OK, boy. Which one of these cards do you want to replace, BOY?"

Is it any wonder Chrysler canceled the project?

So the consultants, now having lost their primary customer, were at a bar one day, and one of them (named L. Ron Hubbard) said: "This nickel-a-line-of-code gig is lame. You know where the real money is at? You start your own religion." And that's how both Extreme Programming and Scientology were born.

The link which explains that Chrysler cancelled the project where a lot of the Extreme Programming and Agile Methodology hype started is on Wikipedia so for all I know that clarification may be gone by the time you read this post. Unfortunately in trying to track down the details in more permanent location all I can find is a USENET thread and more wiki entries. That's pretty interesting, that XP and Agile resulted in a failed software project in the original project where it all started.

There is also some stuff about working at Google where he writes

From a high level, Google's process probably does look like chaos to someone from a more traditional software development company. As a newcomer, some of the things that leap out at you include:

- there are managers, sort of, but most of them code at least half-time, making them more like tech leads.

- developers can switch teams and/or projects any time they want, no questions asked; just say the word and the movers will show up the next day to put you in your new office with your new team.

- Google has a philosophy of not ever telling developers what to work on, and they take it pretty seriously.

- developers are strongly encouraged to spend 20% of their time (and I mean their M-F, 8-5 time, not weekends or personal time) working on whatever they want, as long as it's not their main project.

- there aren't very many meetings. I'd say an average developer attends perhaps 3 meetings a week, including their 1:1 with their lead.

- it's quiet. Engineers are quietly focused on their work, as individuals or sometimes in little groups or 2 to 5.

- there aren't Gantt charts or date-task-owner spreadsheets or any other visible project-management artifacts in evidence, not that I've ever seen.

- even during the relatively rare crunch periods, people still go get lunch and dinner, which are (famously) always free and tasty, and they don't work insane hours unless they want to.

For some reason this reminds me of Malcolm Gladwell's The Talent Myth which is excerpted below

This "talent mind-set" is the new orthodoxy of American management. It is the intellectual justification for why such a high premium is placed on degrees from first-tier business schools, and why the compensation packages for top executives have become so lavish. In the modern corporation, the system is considered only as strong as its stars, and, in the past few years, this message has been preached by consultants and management gurus all over the world. None, however, have spread the word quite so ardently as McKinsey, and, of all its clients, one firm took the talent mind-set closest to heart. It was a company where McKinsey conducted twenty separate projects, where McKinsey's billings topped ten million dollars a year, where a McKinsey director regularly attended board meetings, and where the C.E.O. himself was a former McKinsey partner. The company, of course, was Enron.
...
"We had these things called Super Saturdays," one former Enron manager recalls. "I'd interview some of these guys who were fresh out of Harvard, and these kids could blow me out of the water. They knew things I'd never heard of." Once at Enron, the top performers were rewarded inordinately, and promoted without regard for seniority or experience. Enron was a star system. "The only thing that differentiates Enron from our competitors is our people, our talent," Lay, Enron's former chairman and C.E.O., told the McKinsey consultants when they came to the company's headquarters, in Houston.
...
Among the most damning facts about Enron, in the end, was something its managers were proudest of. They had what, in McKinsey terminology, is called an "open market" for hiring. In the open-market system--McKinsey's assault on the very idea of a fixed organization--anyone could apply for any job that he or she wanted, and no manager was allowed to hold anyone back. Poaching was encouraged. When an Enron executive named Kevin Hannon started the company's global broadband unit, he launched what he called Project Quick Hire. A hundred top performers from around the company were invited to the Houston Hyatt to hear Hannon give his pitch. Recruiting booths were set up outside the meeting room. "Hannon had his fifty top performers for the broadband unit by the end of the week," Michaels, Handfield-Jones, and Axelrod write, "and his peers had fifty holes to fill." Nobody, not even the consultants who were paid to think about the Enron culture, seemed worried that those fifty holes might disrupt the functioning of the affected departments, that stability in a firm's existing businesses might be a good thing, that the self-fulfillment of Enron's star employees might possibly be in conflict with the best interests of the firm as a whole.

Interesting juxtaposition, huh? I've talked to people who've come to Microsoft from Google (e.g. Danny Thorpe) and it definitely is as chaotic as it sounds there. For some reason, the description of life at Google by Steve Yegge reminds me a bit of Microsoft where there were two huge money making projects (Office & Windows in the case of Microsoft and AdWords & AdSense in the case of Google) and then a bunch of good to mediocre projects full of smart people dicking around. Over the years I've seen a reduction of the 'smart people dicking around' type projects over here and more focus on shipping code. I suspect that it's just a matter of time before the same thing will happen at Google as investors seek a better return on their investments once they hit their growth limits in the online advertising space.

There's just one more thing that Steve Yegge wrote that I want to comment on, which is

The thing that drives the right behavior at Google, more than anything else, more than all the other things combined, is gratitude. You can't help but want to do your absolute best for Google; you feel like you owe it to them for taking such incredibly good care of you.

I remember interning at Microsoft five years ago and hearing someone say how grateful he was for "the things Microsoft has done for me" and thinking how creepy and cult-like that sounded. A company pays you at worst 'what they think they can get away with' and at best 'what they think you are worth', neither of these should inspire gratitude. Never forget that or else you'll be on the road to heartbreak.


 

September 27, 2006
@ 08:00 PM

Kevin Briody has an excellent analysis of the recent move by Facebook to branch out beyond the college crowd in his post entitled where he writes

I actually started writing this a few weeks back, when news first broke of Facebook branching out from an exclusive membership model, presumably to take on that Wild West of social networks, MySpace. But given the news that this move is a reality, today, thought I would throw my commentary into the mass that is no doubt being written.

This is a Bad Idea. A classic example of inappropriately twisting a business model to justify investor demands and market expectations.

As I have said before, I love the original Facebook business model. It takes a reasonably cool set of social networking features and a clean UI, and marries it with a hyper-social, hyper-active, local, offline set of communities, i.e. college students clustered on their real-world campuses.
...
For that community, Facebook didn’t force them to separate their social interaction between online and offline - it married the two in a nearly seamless manner.

The site became a way of life for US college students, and even faculty, with use and stickiness numbers that would make most Web-based businesses drool. Facebook nailed the campus social networking experience and achieved fantastic audience penetration, and needed to take the next step.

That next step should have been development of their monetization strategy (deeply targeted advertising for US college students). Focusing on revenue and profit, while continuing to enhance the user experience of their core market.

I agree 100% with what Kevin has written. The Facebook folks have a lock on a very lucrative audience and should be spending cycles figuring out how to (i) keep this audience happy and (ii) monetize such a juicy demographic instead of trying to justify their billion dollar valuation by going up against the MySpaces and Friendsters of the world. Now they have to deal with the potential backlash from their core customer basse as well as the fragmentation of their user base. These are both tough problems to deal with if they crop up.

I suspect the best thing that could happen is for this to fizzle like the move to open up the site to corporations. Otherwise, things could get ugly. I wish them luck, especially now that I know at least one person who works there.


 

Via Greg Linden's blog post entitled R.I.P. Froogle? I found out the CBS MarketWatch article entitled Google's latest could be aimed at eBay listings which informs us that

In a report for Bear Stearns clients, analyst Robert Peck described a new feature that's based on Google Base, a feature Google launched last October that lets people freely list items for sale.

Simultaneously, Google intends to "de-emphasize" its own Froogle shopping search engine, a Web site featuring paid listings from eBay and other online retailers. Google intends for Froogle to no longer be a standalone Web site; instead its listings would be absorbed by other search features, Peck wrote in his report.

There are a number of things I find interesting about this decision [if it is true]. The first point of interest is that this is another step by Google to move from a world where they crawl the Web to where content is submitted to them to be added to their search index directly. This furthers the trend started by offerings such as Google Webmaster tools (formerly Google Sitemaps) and Google Base. Another point of interest is that it seems Google considers product search to not be its own vertical but instead something that should be included automatically in search results via Instant Answers-type functionality (e.g. search for "movies 98052" exposes entry point into movie search page).

Finally, it looks like this is will be the first instance of Google killing a product that was out of beta highlighted on their main page. What product do you think they should kill next? Here's my list of top 3 Google services that should be sent to the product grave yard

  1. Google Reader: I remember how scared people were when this service first debutted, however it's slow and unintuitive UI has made it unusable by anyone except die hard Google fans. It's a testament to its crappiness that it doesn't even make the top 20 list of aggregators used by TechCrunch readers which means that even the early adopter crowd is shunning it.

  2. Google Video: Am I the only one that thinks that this entire service should be replaced with a "site:www.youtube.com" based search similar to what they've done with http://www.google.com/microsoft? The site has a decent search engine but almost every other part of the video sharing experience on the service is subpar. Then again, Youtube has set the bar very high.

  3. Orkut: I was talking with a fellow Microsoft employee last week about the fact that when it comes to community sites, it takes more than features for a site to become popular, it takes people. Sites like Facebook, MySpace and even Friendster managed to be chosen by the right set of influencers and connectors to make them hit critical mass. This never happened with Orkut and it never will [except in Brazil]. My suggestion would be to fold the features of Orkut into Blogger, perhaps as a new service as the SixApart folks have done with Vox (aka LiveJournal + TypePad)

That's my list. I'm sure y'all have reasons to agree or disagree, holla at me in the comments. Also, turn about is fair play so if you want to create a list of services you think Yahoo! and Microsoft should kill for similar reasons I'd definitely find it an interesting read.


 

September 13, 2006
@ 11:14 PM

According to Mike Arrington over at TechCrunch in the blog post entitled Major Google/Intuit Partnership there has been yet another major distribution/bundling deal between Google and a major software distributor. Mike writes

The Google services will be built into QuickBooks 2007, available this Fall, for U.S. customers only.

I sure hope there’s an easy way to turn this stuff off.

Update:
Notes from Analyst call:

Eric Schmidt is talking about embracing the long tail of small businesses on the conference call. Less than half of Quickbooks businesses have an online presence. This will help them get online, he says. Businesses will be able to create an adwords account using pre-filled information from Quickbooks. If the business doesn’t have a website Google will create a notecard page for them. All businesses will be given a $50 credit to start. Google will also create a business listing for businesses for search on Google.com and Google Maps.

Intuit is also integrating Google Desktop (borderline Spyware) into Quickbooks. Thank God this is opt-in…but given that Quicken’s customers are not on average very web savvy, there is a very good chance that many small businesses will opt in without really understanding what they are doing (storing the contents of their hard drive on Google’s servers).

Google has been on a impressive rampage of distribution deals over the past year. It's made deals with AOL, Sun, Adobe, MySpace, Dell and now Intuit to distribute its software and services. This means it'll be even tougher for competitors like Yahoo and Microsoft to gain marketshare from it since it is buying up all the defaults and entry points into search and related services it can find.

A cunning yet expensive strategy. It'll be interesting to see how many more deals they'll make before their done locking up all the defaults they want.


 

Pete Cashmore has a blog post entitled MySpace: We’ll Crush YouTube where he writes

As if we needed more proof of MySpace’s intolerance of outside development, News Corp. chief operating officer Peter Chernin told investors at an industry conference today that since much of YouTube’s traffic comes from MySpace, it’s time to cut out the middle man. Chernin estimated that around 60-70% of YouTube’s visitors come from MySpace, and as a result he wants to ramp up MySpace Video, giving users less incentive to look elsewhere. But while the 60% figure might not be totally accurate, MySpace is definitely a catalyst for YouTube’s growth - they actually reported a dip in traffic when MySpace temporarily disabled YouTube embeds at the end of 2005 (they were forced to allow them again after a user revolt).

But it’s not just YouTube - Chernin expressed distaste for all the services that are feeding the MySpace beast: “If you look at virtually any Web 2.0 application, whether its YouTube, whether it’s Flickr, whether it’s Photobucket…almost all of them are really driven off the back of MySpace, there’s no reason why we can’t build a parallel business.” While I’m not convinced that Flickr gained success on the back of MySpace, Photobucket almost certainly did - ImageShack isn’t mentioned, but it also owes much of its success to America’s leading social network.

The sentiments indicate a very worrying trend: MySpace has stated its intention to clone the best tools, and Chernin believes that MySpace can equal or better the third party tools with in-house products.

Marshall Kirkpatrick sums things up in a post entitled MySpace: We don’t need Web 2.0 on TechCrunch where he states

To summarize: the COO of News Corp. says that Web 2.0 is leaching traffic off of MySpace, that they can build their own services to compete with any of it and that there’s going to be an increasingly aggresive commercial push on the site. That sounds both dangerously arrogant and like a real validation of fears that MySpace dependency is too risky for outside developers.

Om Malik had a piece in Business 2.0 yesterday titled Suddenly Everything’s Coming Up Widgets, where he said “Everyone’s a winner here: MySpace, because it becomes stickier; YouTube and Slide, because they get the traffic; and the user, because he or she gets it all on one page.” It sounds like MySpace’s owners may not want to play a game where everyone wins.

I find this quite surprising. One of the reasons MySpace is where it is today primarily because they stumbled upon the fact that providing a platform for gadgets/widgets allows for richer end user experiences than opening up APIs for viewing and creating content via Atom/RSS/MetaWeblog API which is what blogging sites have traditionally done when building a platform. To turn on the very developers and partner companies that are improving the user experience of your service seems like the road to folly. It's one thing to provide compting experiences to them and quite another to view them as leeches. At Microsoft, we know that the road to success is by building the best platform not just by building the best or most integrated applications. This is a lesson that history has borne out. 

This should be a blow to all those VC funded startups whose entire business model is building MySpace widgets. Not to worry, they can always switch to writing gadgets for Windows Live Spaces. ;)


 

Mike Arrington of TechCrunch has a blog post entitled Facebook Users Revolt, Facebook Replies where he writes

There has been an overwhelmingly negative public response to Facebook’s launch of two new products yesterday. The products, called News Feed and Mini Feed, allow users to get a quick view of what their friends are up to, including relationship changes, groups joined, pictures uploaded, etc., in a streaming news format. Many tens of thousands of Facebook users are not happy with the changes. Frank Gruber notes that a Facebook group has been formed called “Students Against Facebook News Feed”. A commenter in our previous post said the group was closing in on 100,000 members as of 9:33 PM PST, less than a day after the new features were launched. There are rumors of hundreds of other Facebook groups calling for a removal of the new features.

A site calling to boycott Facebook on September 12 has also been put up, as well as a petition to have the features removed. Other sites are popping up as well. There seems to be no counterbalancing group or groups in favor of the changes.

Facebook founder and CEO Mark Zuckerberg has responded personally, saying “Calm down. Breathe. We hear you.” and “We didn’t take away any privacy options.”

I gave the new features a thumbs up yesterday and stick by my review. No new information is being made available about users. Facebook privacy settings remain in their previous state, meaning you can have your information available throughout the network or just among your closest friends. Don’t want a particular piece of information to be syndicated out even to them? Remove any single piece of data by simply clicking the “x” button next to it and it will not appear in the news feed.

If this feature had been part Facebook since the beginning, their users would be screaming if Facebook tried to remove it. It’s a powerful way to quickly get lots of information about people you care about, with easy settings to remove that information for privacy reasons. No one can see anything that they couldn’t see yesterday. It’s just being distributed more efficiently.

I agree that the main problem with the feature is that it is “new” as opposed to any privacy implications. We’ve faced similar problems when designing some of the features of http://spaces.live.com and my advice to the Facebook team would be that it may be better to allow people to opt out of being in feeds than to argue with users about whether it is a privacy violation or not.

That’s a battle that they are not likely to win. Better to be seen as respecting your users wishes as opposed to being paternalistic overlords who think they know what's best for them. Don't make the same mistake Friendster made with fakesters.


 

I just read a blog post by Evan Williams, founder of Blogger entitled Pageviews are Obsolete where he writes

But it's this pageviews part that I think needs to be more seriously questioned. (This is not an argument that Blogger is as popular as MySpace—it's not.) Pageview counts are as suseptible as hit counts to site design decisions that have nothing to do with actual usage. As Mike Davidson brilliantly analyzed in April, part of the reason MySpace drives such an amazing number of pageviews is because their site design is so terrible.

As Mike writes: "Here's a sobering thought: If the operators of MySpace cleaned up the site and followed modern interface and web application principles tomorrow, here's what the graph would look like:"



Mike assumes a certain amount of Ajax would be involved in this more-modern MySpace interface, which is part of the reason for the pageview drop. And, as the Kiko guys wrote in their eBay posting, their pageview numbers were misleading because the site was built with Ajax. (Note: It's really easy to track Ajax actions in Google Analytics for your own edification.)

I've seen a lot of people repeat these claims about MySpace's poor design leading to increased page views. After taking a glance at the average number of page views per user on Alexa for MySpace (38.4 pageviews a day) and comparing it with competing sites such as FaceBook (28.2 pageviews a day), Bebo (31 pageviews a day) and Orkut (38.6 pageviews a day), their numbers don't seem out of the ordinary to me especially if you factor in the sites popularity.

Recently my girlfriend created a MySpace profile and a space on Windows Live Spaces which led me to consider the differences between how both sites are organized. After talking to her for a while about her experiences on both sites it became clear to me that there were fundamental differences in how the sites were expected to be used. Windows Live Spaces concentrates a lot on content creation and sharing that content with people you know (primarily your Windows Live Messenger buddies). On the other hand, MySpace is organized a lot around getting you to "people watch" and explore different user profiles and spaces. Comparing the experience after signing into both services is illuminating.

Anyway, the key observation here is that social networking sites such as MySpace are page view generating engines. Whereas blogging sites such as Blogger and to a lesser extent Windows Live Spaces are less about encouraging people to browse and explore other users on the site and are more about a single user creating content or other users consuming content from a single user. Go ahead and compare both of my girlfriend's spaces and see which one encourages you to click on other users more and which one is more about the owner of the site sharing their [written or digital media] content with you. 

Think about that the next time you hear someone say MySpace gets a lot of page views because they don't use AJAX.


 

Greg Linden has a blog post entitled Google Personalized Search and Bigtable where he writes

One tidbit I found curious in the Google Bigtable paper was this hint about the internals of Google Personalized Search:
Personalized Search generates user profiles using a MapReduce over Bigtable. These user profiles are used to personalize live search results.
This appears to confirm that Google Personalized Search works by building high-level profiles of user interests from their past behavior.

I would guess it works by determining subject interests (e.g. sports, computers) and biasing all search results toward those categories. That would be similar to the old personalized search in Google Labs (which was based on Kaltix technology) where you had to explicitly specify that profile, but now the profile is generated implicitly using your search history.

My concern with this approach is that it does not focus on what you are doing right now, what you are trying to find, your current mission. Instead, it is a coarse-grained bias of all results toward what you generally seem to enjoy.

This problem is worse if the profiles are not updated in real time.

I totally disagree with Greg here on almost every point. Building a profile of a user's interests to improve their search results is totally different from improving their search results in realtime. The former is personalized search while the latter is more akin to clustering of search results. For example, if I search for "football", a search engine can either use the fact that I've searched for soccer related terms in the past to bubble up the offical website of Fédération Internationale de Football Association (FIFA) instead of the National Football League (NFL) website in the search results or it could cluster the results of the search so I see all the options. Ideally, it should do both. However, expecting that my profile is built in realtime (e.g. learning from my search results from five minutes ago as opposed to those from five days ago) although ideal doesn't seem to me to be necessary to be beneficial to end users. This seems like one of those places where a good enough offline-processing based solution is better than a over better engineered real-time solution. Search is rarely about returning or reacting to realtime data anyway. :) 

PS: I do think it's quite interesting to see how many Google applications are built on BigTable and MapReduce. From the post Namespaced Extensions in Feeds it looks like Google Reader is another example.


 

August 31, 2006
@ 07:08 PM

In another episode of the "Google is the new Microsoft" meme, I've been amused to see some VCs brag about how they plan to not invest in any company that potentially competes with Google in any space. Below are two examples I've noticed so far, I'm sure there are more that I've missed

In his blog post entitled The Kiko Affair, Paul Graham writes

Google may be even more dangerous than Microsoft, because unlike Microsoft it's the favorite of technically minded users. When Microsoft launched an application to compete with yours, the first users they'd get would alway be the least sophisticated-- the ones who just used whatever happened to be already installed on their computer. But a startup that tries to compete with Google will have to fight for the early adopters that startups can ordinarily treat as their birthright.
...
The best solution for most startup founders would probably be to stay out of Google's way. The good news is, Google's way is narrower than most people realize. So far Google only seems to be good at building things for which Google employees are the canonical users. That's because they develop software by using their own employees as their beta users
...
They tried hard; they made something good; they just happened to get hit by a stray bullet. Ok, so try again. Y Combinator funded their new idea yesterday. And this one is not the sort of thing Google employees would be using at work. In fact, it's probably the most outrageous startup idea I've ever heard. It's good to see those two haven't lost their appetite for risk.

In his blog post entitled Thoughts on Google Apps, Paul Kedrosky writes

Finally, and this is mostly directed at people sending "Enterprise 2.0" business plans my way: If you're thinking of doing something squarely in Google's enterprise-lusting aim you need to ask yourself one question only: Why? What makes you think that you can do it so much better than Google can that the inevitable free Google Apps product doesn't kick your ass out of the office market? I'm not saying it's impossible, and there are plenty of things outside Google's aim -- including apps that are much more social by design than what Google builds -- but the gate is 99% closed for bringing vanilla,mass-market office apps to the web.

I guess these VCs are planning to stop investing in software companies since Google seems to be increasingly involved in almost every category of software products. I thought the entire point of being a VC was accepting the element of risk involved?


 

August 28, 2006
@ 05:07 PM

I was surprised by the contents of two blog posts I read over the weekend on the same topic. In his post Web 2 bubble ain’t popped yet: Kiko sells for $258,100 Robert Scoble writes

How many employees did Kiko have again? Three, right? Well, they just sold their “failure” for $258,100. Not too shabby!

On the same topic, Om Malik writes in his post Kiko Sells for $258,100

The company recently became talk of the blogs, when the founders decided to cut their losses, and put the company on sale on eBay. Niall and I devoted a big portion of our latest podcast, Snakes on a Business Plan to the Kiko affair. Well, the auction just closed and brought in $258,100. A tidy sum! This explains why Paul was smiling today at The FOO Camp <img alt=" src="http://gigaom.com/wp-includes/images/smilies/icon_wink.gif"> Apparently, Kiko’s angel round was $50,000 in convertible debt, and this sale should cover that. Graham’s YCombinator which did the seed round could come out ahead as well.

I'm confused as to how anyone can define this as good. After you take out however much the investors get back after investing $50,000 there really isn't much left for the three employees to split especially when you remember that one of the things you do as the founder of a startup is not pay yourself that much. At best I can see this coming out as a wash (i.e. the money made from the sale of Kiko is about the same as if the founders had spent the time getting paid working for Google or Yahoo! as full time employees) but I could be wrong. I'd be surprised if it was otherwise.


 

One of the biggest surprises for me over the past year is how instead of Sun or IBM, it's Amazon that looks set to become the defining face of utility computing in the new millenium. Of course, the other shoe dropped when I read about Amazon Elastic Compute Cloud (Amazon EC2) which is described below

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.

Just as Amazon Simple Storage Service (Amazon S3) enables storage in the cloud, Amazon EC2 enables "compute" in the cloud. Amazon EC2's simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon's proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use.

All Amazon needs to do is to add some SQL-like capabilities on top of Amazon S3 and I can't see any reason why any self respecting startup would want to host their own datacenters with the high bandwidth, staff, server, space and power costs that it entails. Anecdotes such as the fact that SmugMug is now storing 300 terabytes of data on Amazon's servers for cheaper than they could themselves will only fuel this trend. I definitely didn't see this one coming but now that it is here, it seems pretty obvious. Companies like IBM and Sun, simply don't have the expertise at building something that has to handle traffic/capacitye at mega-service scale yet be as cheap as possible. Companies like Yahoo!, MSN/Windows Live and Google have this expertise but these are competitive advantages that they likely won't or can't give away for a variety of reasons. However Amazon does have he expertise at building a mega-scale service as cheaply as possible as well as the experience of opening it up as a platform for other companies to build businesses on. With the flood of startups looking to build out services cheaply due to the "Web 2.0" hype, this is a logical extension of Amazon's business of enabling companies to build eCommerce businesses on their platform.

With a few more high profile customers like SmugMug, Amazon could easily become the "dot in dotcomm Web 2.0". Of course, this means that like Sun was during the 90s they'll be pretty vulnerable once the bubble pops.


 

It looks like the big news this morning is that Google just announed Google Apps for your Domain. From the press release Google Launches Hosted Communications Services we learn

Google Apps for Your Domain, an expansion of the Gmail for Your Domain service that launched in February 2006, currently includes Gmail web email, the Google Talk instant messaging and voice calling service, collaborative calendaring through Google Calendar, and web page design, publishing and hosting via Google Page Creator. Domain administrators use a simple web-based control panel to manage their user account list, set up aliases and distribution lists, and enable the services they want for their domain. End users with accounts that have been set up by their administrator simply browse to customized login pages on any web-connected computer. The service scales easily to accommodate growing user bases and storage needs while drastically reducing maintenance costs.

Google will provide organizations with two choices of service.

  • A standard edition of Google Apps for Your Domain is available today as a beta product without cost to domain administrators or end users. Key features include 2 gigabytes of email storage for each user, easy to use customization tools, and help for administrators via email or an online help center. Furthermore, organizations that sign up during the beta period will not ever have to pay for users accepted during that period (provided Google continues to offer the service).
  • A premium version of the product is being developed for organizations with more advanced needs. More information, including details on pricing, will be available soon.

If this sounds familiar to you, that's because it is. This is pretty much the same sales pitch as Microsoft's Office Live. Right down to having tiered versions that range from free (i.e. Office Live Basics) to paid SKUs for businesses with more 'advanced' needs (i.e. Office Live Essentials). With Google officially entering this space, I expect that the Office Live team will now have some pressure on their pricing model as well as an incentive to reduce or remove some of the limitations in the services they offer (e.g. the fairly low limits on the amount of email addresses one can create per domain).

As usual, the technology blogs are full of the Microsoft vs. Google double standard. When Microsoft announced Office Live earlier this year, the response was either muted or downright disappointed because it wasn't a Web-based version of Microsoft Office. An example of such responses is Mike Arrington's post entitled Microsoft Office Live goes into Beta. On the flip side, the announcement of Google Apps for your Domain which is basically a "me too" offering from Google is heralded by Mike Arrington in his post Google Makes Its Move: Office 2.0 as the second coming of the office suite. The difference in the responses to what are almost identical product announcements is an obvious indication at how both companies are perceived by the technology press and punditry.

I personally prefer Om Malik's take in his post Web Office Vs Microsoft Office where he states

"Web Office should not be about replacing the old, but inventing the new web apps that solve some specific problems".

This is pretty much the same thing I heard Ray Ozzie and Sergey Brin say at last years Web 2.0 conference when they were both asked [on different occassions] about the possibility of replacing desktop office suites with Web-based software. Enabling people in disparate locations to collaborate and communicate is the next major step for office productivity suites. One approach could be replacing everything we have today with Web-based alternatives, the other could be making the desktop software we have today more Web savvy (or "live" if you prefer the Microsoft lingo). I know which one I think is more realistic and more likely to be acceptable to businesses today. What do you think?

My next question is whether Google is going to ship consumer targetted offerings as Microsoft has done with Windows Live Custom Domains or is the free version of Google Apps for your Domain expected to be the consumer version?

Disclaimer: The above statements are my opinions and do not in any way reflect the plans, thoughts, intentions or strategies of my employer.


 

August 25, 2006
@ 12:25 AM

It looks like I'm now writing a Windows Live gadget every week. My latest gadget is a port of the Flickr badge to a Windows Live gadget. It's in the approval pipeline and should show up under the list of gadgets I've written in the next day or so. To get the gadget working, I had to use the Flickr API. Specifically, I used the flickr.people.findByUsername method to convert a username to an internal Flickr ID. Ironically Coincidentally, I had recently read something by Alex Bosworth criticizing this very aspect of the Flickr API in his post How To Provide A Web API where he wrote

Simple also means don’t be too abstract. Flickr for example chooses in its API to require the use of its internal ids for all API calls. This means for example that every call to find information about a user requires a call first to find the internal id of the user. Del.icio.us on the other hand just requires visible names, in fact internal ids are hidden everywhere.

Actually, it's much worse than this. It seems that Flickr is inconsistent in how it maps user names back to internal IDs. For example, take Mike Torres who has 'mtorres' as his Flickr ID. I can access his Flickr photos by going to http://flickr.com/photos/mtorres. When I use the Flickr API explorer for flickr.people.findByUsername and pass in 'mtorres' as the username I get back the following ID; 25553748@N00. When I go to http://flickr.com/photos/25553748@N00 I end up going to some other person's page who seems to be named 'mtorres' as well.

However when I plug "Mike Torres" into the flickr.people.findByUsername method instead of 'mtorres' I get '37996581086@N01' which turns out to be the right ID since going to http://www.flickr.com/photos/37996581086@N01 takes me to the same page as http://flickr.com/photos/mtorres. Weird.

Perhaps this is a naming collision caused by the merging of Flickr & Yahoo! IDs?


 

I was just reading Paul Graham's post entitled The Kiko Affair which talks about the recent failure of Kiko, an AJAX web-calendaring application. I was quite surprised to see the following sentence in Paul Graham's post

The killer, unforseen by the Kikos and by us, was Google Calendar's integration with Gmail. The Kikos can't very well write their own Gmail to compete.

Integrating a calendaring application with an email application seems pretty obvious to me especially since the most popular usage of calendaring applications is using Outlook/Exchange to schedule meetings in corporate environments. What's surprising to me is how surprised people are that an idea that failed in 1990s will turn out any differently now because you sprinkle the AJAX magic pixie dust on it.

Kiko was a feature, not a full-fledged online destination let alone a viable business. There'll be a lot more entrants into the TechCrunch deadpool that are features masquerading as companies before the "Web 2.0" hype cycle runs its course. 


 

Caterina Fake of Flickr has a blog post entitled BizDev 2.0 where she writes

Several companies -- probably more than a dozen -- have approached us to provide printing services for Flickr users, and while we were unable to respond to most of them, given the number of similar requests and other things eating up our time, one company, QOOP, just went ahead and applied for a Commercial API key, which was approved almost immediately, and built a fully-fleshed out service. Then after the fact, business development on our side got in touch, worked out a deal -- and the site was built and taking orders while their competitors were still waiting for us to return their emails. QOOP even patrols the discussions on the Flickr boards about their product, and responds and makes adjustments based on what they read there. Now that's customer service, and BizDev 2.0.

Traditional business development meant spending a lot of money on dry cleaning, animating your powerpoint, drinking stale coffee in windowless conference rooms and scouring the thesaurus looking for synonyms for "synergy". Not to mention trying to get hopelessly overbooked people to return your email. And then after the deal was done, squabbling over who dealt with the customer service. Much, much better this way!

I know exactly where Catrina is coming from. Given that I work on the platform that powers Windows Live Spaces which has over 100 million users and 5.2 billion photos with over 6 million being uploaded daily, I've been on the receiving end of similar conversations about business partnerships revolving around integrating with the blogs, photo albums, lists and user profiles in our service. All of these partnerships have sounded obsolete to me in the age of open APIs. It seems to me to be much better to support de-facto industry standards like the MetaWeblog API that enables any tool or website to integrate with our service than have proprietary APIs that can only be accessed by people who we've made exclusive business deals with us. That seems better for our service and better for our users to me.

This definitely changes the game with regards to how our business development folks approach certain types of business partnerships. I probably wouldn't have called it BizDev 2.0 though. ;) 


 

August 16, 2006
@ 03:45 AM

Today I learned about developers.facebook.com which proclaims

Welcome to Facebook Developers (beta), where you can create your own projects using our application programming interface (API).

In case you've been living under a rock for the past couple of months, The Facebook is like MySpace but for college kids. It seems to have made the transformation from a cool application of the moment (a.k.a. a fad) to an actual must-have utility among the college students I've talked to about it. I've heard college girls say they look guys up on The Facebook as part of pre-date screening and these were sorority girls not geeks. The fact that they are providing an API is a very interesting turn of events especially when you consider their dominant position in their market. 

I'm particularly interested in the Facebook API because I've been thinking about what we should do to expose the Windows Live friends list via an API. The problem with exposing APIs for social networks and contact lists is that the worst case scenario is that your API gets a lot of usage from your competitors (e.g. Zooomr vs. Flickr). I've been thinking about this problem on-and-off for the past couple of months and was interested to see how Facebook API handled this problem or whether, like me, they'd come to the conclusion that if the main use of your API is people trying to leave your service then you've got other problems than just the API. I checked out the definition of the facebook.friends.get method and left with more questions than answers. The API states

facebook.friends.get

Returns the identifiers of the current user's Facebook friends. The current user is determined from the session_key. The values returned from this call are not storable.

Response

The friend ids returned are those friends visible to the calling application. If no friends are found, the method will return an empty result element.

The parts highlighted in red are interesting to say the least. I wonder what exactly is meant by "values returned from this call are not storable". Is this legal wording? Are the values encrypted in some way? What exactly does that mean? It looks like I may need to do some sleuthing around the forums except I don't have a Facebook account. Hmmm...

I was also interested in the authentication model used by the Facebook API. From reading the documentation, their authentication scheme reminds me of Yahoo's Browser Based Authentication Scheme (scheme) in that it requires users to always log-in from a browser and then either be redirected back to the calling page (much like Microsoft's Passport Windows Live ID) or for the target application to re-use the URL it got after the browser was closed if it is a desktop application. Surely, there must be a better way to authenticate desktop applications against online services than habing them launch a Web browser and having a separate, dissonant sign-in process.

PS: If the Facebook API sounds interesting to you and you'd like to do similar things with the Windows Live friends list I'd love to hear what your scenarios are. Holla at me. 


 

A couple of days ago, I wrote that based on my experiences with Kuro5hin I wouldn't be surprised by small sets of users dominating the content and focus of Digg since it is practically the same kind of site. Duggtrends has a blog post entitled Digg user statisitcs & trends which confirms these suspicions. It states

From our database, for the period of 6/19/2006 9:31:28 PM to 7/30/2006 4:41:34 PM a total of 6013 stories were promoted to front page of these
  • top 10 users contributed 1792 i.e 29.8%
  • top 100 contributed 3324 stories i.e 55.28% (which is again what exactly SEOMOZ reported)
This clearly shows the shift from the Kevin Rose reported numbers from 26.8% to 55.28%; top users are contributing more & more to digg

As per Jason Martinez (and Calacanis points in his blog) 60% of Digg’s front page is the top 0.03% users and provides this graph.

It looks like the 1% rule is in full effect. This has always been the nature of online communication from mailing lists and newsgroups to Web-based message boards and social bookmarking sites. Human nature will always shine through at the end.

PS: Has anyone else seen that the Digg REST API has now been documented? Thanks to Michael Brundage for the tip.


 

User interfaces for computers in general and web sites in particular seem to be getting on my nerves these days. It's really hard to browse for what you are looking for on a number of buzzword-complaint websites today. Most of them seem to throw a tag cloud and/or search box at you and call it a day.

Search boxes suck as a navigation interface because they assume I already know what I'm looking for. I went to the Google Code - Project Hosting website and wanted to see the kinds of projects hosted there. Below is a screenshot of the website from a few minutes ago.

Notice that the list of project labels (aka tags) shown below the search box are just 'sample labels' as opposed to a complete classification scheme or hierarchy. They don't even list fairly common programming topics like VisualBasic, Javascript or FreeBSD. If I want to browse any of these project labels, I have to resort to poring over search results pages with minimal information about the projects.

Using tag clouds as a navigation mechanism is even more annoying. I recently visited Yahoo! Gallery to see what the experience was like when downloading new plugins for Yahoo! Messenger. On the main page, there is a link that says Browse Applications which takes me to a page that has the following panel on the right side. So far so good.

I click on the Messenger link and then was taken to the following page.

What I dislike about this page is how much space is taken up by useless crap (i.e. the tag cloud full of uninformative tags) while the actual useful choices for browsing such as 'most popular' and 'highest rated' are given so little screen real estate and actually don't even show up on some screens without scrolling down. The tag cloud provides little to no value on this page except to point out that whoever designed is hip to all the 'Web 2.0' buzzwords.

PS: Before anyone bothers to point this out, I realize a number of Microsoft sites also have similar issues.


 

I just noticed that there is now Project Hosting on Google Code via Slashdot. The Slashdot article has the following rich quote

Joe 'Zonker' Brockmeier sat down for a talk with Greg Stein and Chris DiBona, who say that the product is very similar to sites like SourceForge but is not intended to compete with them. From the article: "Instead, Stein says that the goal is to see what Google can do with the Google infrastructure, to provide an alternative for open source projects. DiBona says that it's a 'direct result of Greg concentrating on what open source projects need. Most bugtrackers are informed by what corporations' and large projects need, whereas Google's offering is just about what open source developers need. Stein says that Google's hosting has a 'brand new look' at issue tracking that may be of interest to open source projects, and says 'nobody else out there is doing anything close to it.'"

Last year, when I saw Chris DiBona announce Google Code at the O'Reilly ETech conference, he said that that they woild work with SourceForge to improve the service. I wonder what happened. The site seems pretty sparse right now, I doubt I'll be moving RSS Bandit from SourceForge anytime soon.


 

A few weeks ago, Matt Griffiths wrote a response to my blog post Yahoo! Working on Open Source GFS Clone? entitled Why would Yahoo support an open source version of the Google File System? where he wrote

So why would Yahoo! do this? Why would they create open source versions of tools that could give them a short-term competitive advantage? I think Joel Spolsky said it best:
Smart companies try to commoditize their products' complements. [Joel Spolsky]
Yahoo! is not in the business of selling software. They sell advertising. Software is one of their biggest complements. The best way to commoditize software is to open source it.
I disagree that software is a complement to what Yahoo! sells which is advertising real estate on their online services such as search, email and news sites. Software is more of an input into the production process (i.e. a means to the end) than a complement of the finished product.

A complement is something that is purchased along with your product to make it useful. If you are a vendor of consumer operating systems, then desktop PCs are your product's complement and you want PCs to be as cheap as possible. If you are a manufacturer of gas guzzling SUVs vendor, then gasoline is your product's complement and you want gas prices to be as low as possible. If you are a software consultant, then shrink-wrap software is your product's complement and you want software licenses to be as cheap as possible. None of these examples is analogous to Yahoo! open sourcing a tool that makes it easier and cheaper for them to build online services and thus have more & better real estate to sell to advertisers.

I can see a couple of explanations for Yahoo! making this move. On the one hand, they could believe that distributed software development isn't their core competency and would like to outsource some of the harder bits by Open Sourcing it. The problem with that approach is that as the Mozilla project has shown, it may take years to get a critical mass of external developers working on the project and it would still need significant contributions from Yahoo! to stay afloat. Another justification, could be that Yahoo! realizes that even with technologies like GFS/Hadoop there is still a lot of hardware expenditure, operations expertise and infrastructure software needed to run a megascale service. These are often beyond the resources of most competitors and for the ones that do have those resources (e.g. Google) they already have similar technologies. Thus there is little to fear of a competitor using Hadoop against them. A third option could be that Hadoop may not be considered to be strategic by Yahoo! management which is why it has been allowed to be open sourced. In that case, management may either be underestimating the importance of technologies like Hadoop or may just truly think that their competitive advantage lies elsewhere.

Bah, I should probably get to work instead of engaging in idle speculation on a Thursday morning. :) 


 

The New York Times has an article entitled In the Race With Google, It’s Consistency vs. 'Wow' which talks about competition between the big four online services (Google/Yahoo/Microsoft/AOL). The article dismisses Microsoft and AOL as also rans, then primarily focuses on competition between Yahoo! and Google. Below are some excerpts from the article

Google is continuing to extend its lead in users and revenue from Web search, while Yahoo’s attempt to compete is foundering. Last week, Yahoo reported weak search revenue and said it would delay a critical search advertising system, sending its shares down 22 percent to a two-year low.

With AOL and MSN from Microsoft losing share and plagued by strategic confusion, Yahoo is in a position to further solidify its lead as the Web’s most popular full-service Internet portal, so any incursions by Google into areas like e-mail and maps are a threat.

“There is a tradeoff between integration and speed,” Mr. Eustace said. “We are living and dying by being an innovative, fast-moving company.” Sometimes this penchant for speed and innovation can cause Google to zoom past the basics. When asked about the lack of an address book in Google Maps in an interview last fall, Marissa Mayer, Google’s vice president for search products and user experience, said it was a gap in the product. She said it was much easier to get the company’s engineers to spend time developing pioneering new technology than a much more prosaic address storage system.

There are risks in each approach. Google tends to introduce a lot of new products and then watch to see what works. This has the potential to alienate users if there are too many half-baked ideas or false starts. At the same time, Yahoo risks being seen as irrelevant if it tries to put so many features into each product that it is always months late to market with any good idea.

“Yahoo has lost its appetite for experimentation,” said Toni Schneider, a former product development executive at Yahoo who is now chief executive of Automattic, a blogging software company. “They used to be a lot more like Google, where someone would come up with a cool idea and run with it.” While Yahoo’s processes have become too bureaucratic, it is still attracting an audience, Mr. Schneider said. “Google’s products may be more innovative, but at the end of the day, Yahoo is pretty good at nailing what the user really wants.”

So far, outside of the Web search business, neither company appears to be able to make a significant dent in the position of the other. Both companies are gaining users as AOL and MSN decline.

Despite the spin on the article, the chart provided seems to show that Microsoft is in the running for the top spots among the various key online services although I'm quite surprised that neither MSN Maps nor Windows Live Local show up in the list of popular mapping sites. In addition, the demographics are different for worldwide usage versus the United States. I believe MSN Spaces and MSN/Windows Live Messenger are at the top of their categories world wide according to comScore.

It is good to see more people pointing out that all the so-called innovation in the world is a waste of time if you don't handle basic user scenarios. It's more important that I don't have to type my address every time I use a mapping website I visit regularly than that it uses AJAX extensively.

It's also interesting to see complaints of bureaucracy at Yahoo! from Toni Schneider (formerly of Oddpost which was acquired by Yahoo) which echo the same comments made by Jeffrey Veen (formerly of MeasureMap acquired by Google) about bureaucracy at Google. I guess that highlights the difference between working at a startup versus working at a big company like Yahoo! or Google. 

I think the framing of the competition between online serves as being about consistency vs. 'Wow' factor may be a straw man. I think it is more about integrated services versus siloed applications. After all, a portal can consistently use AJAX or Flash and still fail to gain traction with users because it doesn't satisfy basic scenarios. On the other hand, when applications allow users to do multiple things at once from a single application then goodness ensues. MySpace is a good example of this, it integrates social networking, photo sharing, blogging, music sharing and more into a single highly successful application. MSN Spaces does the same and is also highly successful. On the flip side, Google has three or four different overlapping websites to do the same thing. That costs you in the long run. Another good example, is Google search in that it provides a single search box yet provides a whole lot more than website search from that box. Depending on your search, it also does music search, map search, currency conversion, metric unit conversions, stock quotes, news search, image search and more.

As Google search and MySpace have shown there's more of a 'Wow' factor when an application takes a well integrated, multi-disciplinary approach than from merely being AJAXy.


 

Given the amount of time I now spend working on features for Windows Live Messenger I've started reading blogs about the IM/VOIP industry such as Skype Journal and GigaOm. I now find news stories that I'd traditionally miss, such as the Skype Journal blog post entitled "We have no interest in cracking, replicating, reverse engineering or pirating Skype's software."which links to a blog entry entitled not rumors about the recent news that the Skype protocol had been reverse engineered. The linked blog post is excerpted below.

Well, rumors are not rumors :) But things are not going like the ways people think, as they are disclosed a little without many further explainations, anecdotes breeds especially when sensitively relating with a big hot biz and politics.

As a long-term friend with this Chinese team, but an outsider in Skype, VOIP or P2P tech and biz, I observed the whole process of that shock and wondered why they did not give a word to declare their status. Because I once heard the beautiful prospect in their minds and know it is not what some people talked about in the Internet. But even then I was confused with blooming gossips. I thought that they just did not realize how a small stone could stir big waves. So I quoted some interesting, constructive (well, I like the open source ideas most), exaggerating and offensive comments and wrote an email to them.

This morning, I received a call from China and then followed an email. In the call, I urgently asked them about that rumors, they did not deny but said they also bothered with endless calls and emails for all purposes - interviews, verifications, legal affairs, biz talks... which disturbed their main aim and daily work- research, and in email they wrote

    "We have no interest in cracking, replicating, reverse engineering or pirating Skype's software. We just want to invent a better one. Having learned from and inspired by Skype, we are going to create a P2P Internet platform where all social groups can enjoy efficient, secure and free communication. This network platform will be better than SkypeNet that we are using today."

Then we chatted about some broad issues to fulfill my curiosity, which mainly related to the (potential) reaction of Skype Corp. They said they are just kids standing on the shoulders of giants.

If this blog post is accurate then it looks like the various pundits claiming that this will lead to a plethora of 3rd party desktop clients using the Skype network are out of luck. Of course, this could still happen if the research team publishes their findings but if they truly are fans of the Skype team they may not want to raise their ire. Either way, it'll be interesting to see what they end up building based on their knowledge of the Skype protocol. 


 

July 15, 2006
@ 10:25 PM

Nathan Torkington has a blog post entitled A Week in the Valley: GData on the O'Reilly Radar blog that talks about the growth of the usage of GData & the Atom Publishing Protocol within Google as well as Marc Lukovsky's take on how this compared to his time at Microsoft working on Hailstorm. Nat writes

They're building APIs to your Google-stored data via GData, and it's all very reminiscent of HailStorm. Mark, of course, was the architect of that. So why's he coming up with more strategies to the same ends? I figure he's hoping Google won't screw it up by being greedy, the way Microsoft did...The reaction to the GData APIs for Calendar have been very positive. This is in contrast to HailStorm, of course, which was distrusted and eventually morphed its way through different product names into oblivion. Noting that Mark's trying again with the idea of open APIs to your personal data, I joked that GData should really be "GStorm". Mark deadpanned, " I wanted to call it ShitStorm but it didn't fly with marketing".

Providing APIs to access and manipulate data owned by your users is a good thing. It extends the utility of the data outside that of the Web applications that may be the primary consumer of the data and it creates an ecosystem of applications that harness the data. This is beneficial to customers as can be seen by looking around today at the success of APIs such as the MetaWeblog API, Flickr API or del.icio.us API.

Five years ago, while interning at Microsoft, I saw a demo about Hailstorm in which a user visiting an online CD retailer was showed an ad for a concert they'd be interested in based on their music preferences in Hailstorm. The thinking here was that it would be win-win because (i) all the user's data is entered and stored in one place which is convenient for the user (ii) the CD retailer can access the user's preferences from Hailstorm and cut a deal with the concert ticket provider to show their ads based on user preferences and (iii) the concert ticket provider gets their ads shown in a very relevant context.

The big problem with Hailstorm is that it assumed that potential Hailstorm partners such as retailers and other businesses would give up their customer data to Microsoft. As expected most of them told Microsoft to take a long walk of a short pier. 

Unfortunately Microsoft didn't take the step of opening up these APIs to its online services such as Hotmail and MSN Messenger but instead quietly canned the project. Fast forward a few years later and the company is now playing catchup to ideas it helped foster. Amusingly, people like Mark Lucovsky and Vic Gundotra who were influential during the Hailstorm days at Microsoft are now at Google rebuilding the same thing.

I've taken a look at GData and have begun to question the wisdom of using Atom/RSS as the baseline for information interchange on the Web. Specifically, I have the same issues as Steven Ickman raised in a comment on DeWitt Clinton's blog where he wrote

From a search perspective I’d argue that the use of either format, RSS or Atom, is pretty much a hack. I think OpenSearch is awesome and I understand the motivators driving the format choices but it still feels like a hack to me.

Just like you I want to see rich structured results returned for queries but both formats basically limit you to results of a single type and contain a few known fields (i.e. link, title, subject, author, date, & enclosure) that are expected to be common across all items.

Where do we put the 100+ Outlook defined contact fields and how do we know that a result is a contact and not an appointment or auction? Vista has almost 1000 properties defined in its schema so how do we convey that much metadata in a loseless way? Embedded Microformats are a great sugestion for how to deal with richer content but it sort of feels like a hack on top of a hack to me? What’s the Microformat for an auction? Do I have to wait a year for some committee to arrive at joint aggreement on what attributes define an auction before I can return structured auction results?

When you have a hammer, everything looks like a nail. It seems Steven Ickman and I reviewed OpenSearch/GData/Atom with the same critical lens and came away with the same list of issues. The only thing I'd change in his criticism is the claim that both formats (RSS & Atom) limit you to results of a single type, that isn't the case. Nothing stops a feed from containing data of wildly varying types. For example, a typical MSN Spaces RSS feed contains items that represent blog posts, photo albums, music lists, and book lists which are all very different types.

The inability to represent hierarchical data in a natural manner is a big failing of both formats. I've seen the Atom Threading Extensions but that seems to be a very un-XML way for an XML format to represent hierarchy. Especially given how complicated message threading algorithms can be for clients to implement.

It'll be interesting to see how Google tackles these issues in GData.


 

The Google Adwords API team has a blog post entitled Version 3 Shutdown Today which states

Please take note… per our announcement on May 12, we will shutdown Version 3 of the API today.

Please make sure you have migrated your applications to Version 4 in order to ensure uninterrupted service. You can find more information about Version 4 (including the release notes) at http://www.google.com/apis/adwords/developer/index.html.

-- Rohit Dhawan, Product Manager

This is in compliance with the Adwords API versioning policy which states that once a new version of the WSDL for the Adwords API Web service is shipped, the old Web service end point stops being supported 2 months later. That's gangsta.

Thanks to Mark Baker for the link.


 

Tim O'Reilly has a blog post entitled Operations: The New Secret Sauce where he summarizes an interview he had with Debra Chrapaty, the VP of Operations for Windows Live. He writes

People talk about "cloud storage" but Debra points out that that means servers somewhere, hundreds of thousands of them, with good access to power, cooling, and bandwidth. She describes how her "strategic locations group" has a "heatmap" rating locations by their access to all these key limiting factors, and how they are locking up key locations and favorable power and bandwidth deals. And as in other areas of real estate, getting the good locations first can matter a lot. She points out, for example, that her cost of power at her Quincy, WA data center, soon to go online, is 1.9 cents per kwh, versus about 8 cents in CA. And she says, "I've learned that when you multiply a small number by a big number, the small number turns into a big number." Once Web 2.0 becomes the norm, the current demands are only a small foretaste of what's to come. For that matter, even server procurement is "not pretty" and there will be economies of scale that accrue to the big players. Her belief is that there's going to be a tipping point in Web 2.0 where the operational environment will be a key differentiator
...
Internet-scale applications are really the ones that push the envelope with regard not only to performance but also to deployment and management tools. And the Windows Live team works closely with the Windows Server group to take their bleeding edge learning back into the enterprise products. By contrast, one might ask, where is the similar feedback loop from sites like Google and Yahoo! back into Linux or FreeBSD?

This is one of those topics I've been wanting to blog about for a while. I think somewhere along the line at MSN Windows Live we realized there was more bang for the buck optimizing some of our operations characteristics such as power consumption per server, increasing the number of servers per data center, reducing cost per server, etc than whatever improvements we could make in code or via database optimizations. Additionally, it's also been quite eye opening how much stuff we had to roll on our own which isn't just standard parts of a "platform". I remember talking to a coworker about all the changes we were making so that MSN Spaces could be deployed in multiple data centers and he asked why we didn't get this for free from "the platform". I jokingly responded "It isn't like the .NET Framework has a RouteThisUserToTheRightDataCenterBasedOnTheirGeographicalLocation() API does it?".

I now also give mad props to some of our competitors for what used to seem like quirkiness that now is clearly a great deal of operational savviness. There is a reason why Google builds their own servers, when I read things like "One-third of the electricity running through a typical power supply leaks out as heat" I get quite upset and now see it as totally reasonable to build your own power supplies to get around such waste. Unfortunately, there doesn't seem to be a lot of knowledge out there about the building and managing a large scale, globally distributed server infrastructure. However we are feeding a lot of our learnings back to the folks building enterprise products at Microsoft (e.g. our team now collaborates a lot with the Windows Communication Foundation team) as Debra states which is great for developers building on Microsoft platforms. 


 

Greg Linden has a blog post entitled Yahoo building a Google FS clone? where he writes

The Hadoop open source project is building a clone of the powerful Google cluster tools Google File System and MapReduce.

I was curious to see how much Yahoo appears to be involved in Hadoop. Doug Cutting, the primary developer of Lucene, Nutch, and Hadoop, is now working for Yahoo but, at the time, that hiring was described as supporting an independent open source project.

Digging further, it seems Yahoo's role is more complicated. Browsing through the Hadoop developers mailing list, I can see that more than a dozen people from Yahoo appear to be involved in Hadoop. In some cases, the involvement is deep. One of the Yahoo developers, Konstantin Shvachko, produced a detailed requirement document for Hadoop. The document appears to lay out what Yahoo needs from Hadoop, including such tidbits as handling 10k+ nodes, 100k simultaneous clients, and 10 petabytes in a cluster.

Also noteworthy is Eric Baldeschwieler, a director of software development at Yahoo, who recently talked about direct support from Yahoo for Hadoop. Eric said, "How we are going to establish a testing / validation regime that will support innovation ... We'll be happy to help staff / fund such a testing policy."

I find this effort by Yahoo! to be rather interesting given that platform pieces like GFS, BigTable, MapReduce and Sawzall give Google quite the edge in building mega-scale services and in Greg Linden's words are 'major force multipliers' that enable them to pump out new online services at a rapid pace. I'd expect Google's competitors to build similar systems and keep them close to their chest not give them away. I suspect that the reason Yahoo! is going this route is that they don't have enough folks to build this in-house and have thus collaborated with Hadoop project to get some help. This could potentially backfire since there is nothing stopping small or large competitors from reusing their efforts especially if it uses a traditional Open Source license.

On a related note, Greg also posted a link to an article by David F. Carr entitled How Google Works which has the following interesting quote

Google has a split personality when it comes to questions about its back-end systems. To the media, its answer is, "Sorry, we don't talk about our infrastructure."

Yet, Google engineers crack the door open wider when addressing computer science audiences, such as rooms full of graduate students whom it is interested in recruiting.

As a result, sources for this story included technical presentations available from the University of Washington Web site, as well as other technical conference presentations, and papers published by Google's research arm, Google Labs.

I do think it is cool that Google developers publish so much about the stuff they are working on. One of the things I miss from being on the XML team at Microsoft is being around people with a culture of publishing research like Erik Meijer and Michael Rys. I even got a research paper on XML query languages published while on the team. I'd definitely would like to publish research quality papers on some of the stuff I'm working on now. I've done MSDN articles and a ThinkWeek paper in the past few years, it's probably about time I start thinking about writing a research paper again. 

PS: If you work on online services and you don't read Greg Linden's blog, you are missing out. Subscribed. 


 

June 30, 2006
@ 04:28 AM

It seems the Web API authentication discussion has been sparked up all over the Web by the various announcements of Windows Live ID and the Google Account Authentication for Web apps . In his blog post Google's authentication vs. Microsoft's Live ID Eric Norlin writes

Recent announcements of Google's authentication service have prompted comparisons to Passport, and even gotten to Dick Hardt (of "Identity 2.0" fame) to call it the, "deepening of the identity silo." I'd like to contrast Google's work with Microsoft's recent work around Live ID.

Microsoft's Live ID *is* the old Passport — with a few key changes. Kim Cameron's work around the identity metasystem has driven the concept of InfoCards (now called CardSpace) deep inside of Microsoft. In essence, Kim's idea is that there is a "metasystem" which utilizes WS-Trust to translate tokens, so that all identity systems can interact with each other.

Of extreme importance is the fact that Windows Live ID will support WS-Trust, WS-Federation, CardSpace and ADFS (active directory federation server). This means that A) Windows Live ID can interact with other identity metasystem implementations (Open Source versions, for example); B) that your corporate active directory environment can be federated into Windows Live ID; and C) the closed system that was Passport has now effectively been transformed into an open (standards-based) and transparent system that is Live ID.

Contrast all of this with Google's announcement: create Google account, store user information at Google, get authentication from Google — are we sensing a trend? While Microsoft is now making it easy to interact with other (competing) identity systems, Google is making it nearly impossible. All of which leads one to ask - why?

Perhaps it's because there are now so many old-school Microsoft people at Google? ;)

On a more serious note, I suspect that the Google folks simply didn't think about the federation angle when designing the authentication model for their APIs as opposed to this being some 'evil plot' by Google to create an identity silo.


 

One of my coworkers sent me a link to the blog post PhotoBucket Leads Photo Sharing Sites; Flickr at #6 on the HitWise company blog. The highlights of the post are excerpted below

In the SF tech bubble that I live in, most of the talk about photo sites has been centered on Flickr. In fact, you could get the impression from most people I meet that Flickr is the ONLY site at which you can share and store photos. Examination of the category however, shows that Flickr is #6 among the top 10 photo sharing sites, with a market share of 5.95%. Industry standbys like Yahoo! Photos, Webshots Community, and Kodak Gallery currently rank higher than Flickr.

blog062106.jpg

Photobucket dominates the category, with a 44% market share. It surpassed Yahoo! Photos in January, and its share of visits increased by 34% in the four months from February 2006 to May 2006. Flickr, my friends should be happy to note, has also been growing rapidly, increasing 44% in the past four months, and up from a rank of #9 in this category one year ago (week ending 6/18/05). Slide has also taken off this spring, with its visits increasing more than ten fold in the past four months.

In the comments, someone asked about the methodology and why the HitWise marketshare numbers differ significantly from those of other ratings companies like ComScore. The response is that the HitWise numbers are based on page views from sampling 10 million users while ComScore numbers are for unique users. According the the comment ComScore states that both Flickr and Photobucket  get about 16.5 million unique visitors a month while Yahoo! Photos gets about twice that number. Besides the page views versus unique users distinction, another thing that makes this somewhat of an apples to oranges comparison is that the HitWise sample is for Internet users in the USA while ComScore is talking about worldwide usage.

However this is still an instructive set of statistics. For one it shows that even if Flickr does have more people talking about it in the tech blogosphere, Photobucket is generating a lot more page views [mainly through MySpace integration]. Secondly, it shows the value of an integrated suite of social applications [photo sharing, social networking, video sharing, blogging, etc] in engaging users. Flickr may or may not have just as many visitors as Photobucket but it's clear that the average visitor to Photobucket views significantly more photos from the site than the average Flickr visitor. This is primarily due to the fact that Photobucket is one of the top image hosting sites used by MySpace users. This seems to validate the approach of building an integrated social software application like MSN Spaces or Yahoo! 360 instead of a mishmash of narrowly tailored social software applications. Where I think Microsoft and Yahoo! have gone wrong and MySpace has gotten it right is that they rely a lot on an ecosystem of supporting sites for extra features (e.g. image hosting, video hosting, etc) instead of trying to do it all in-house. This enables them to innovate a lot faster since Microsoft and Yahoo! are then competing with multiple companies instead of just one.


 

It seems that during the storm of news and blog posts about Robert Scoble and Bill Gates, I missed a couple of newsworthy events involving my employer. Last week Adobe released a press release entitled Adobe and Microsoft which is excerpted below

Adobe has been in discussions with Microsoft for some time, sharing concerns regarding features and functionality in the upcoming releases of Vista (the next version of Microsoft’s operating system) and Microsoft Office. While much of the press coverage to date has centered on disputes over PDF and XPS (a competitive technology to PDF) in MS Office and Vista, the real issue is the protection of open standards.

Adobe is committed to open standards. Adobe publishes the complete PDF specification and makes it available for free, without restrictions, without royalties, to anyone who cares to use it. Because we license the PDF specification so openly, it has become a de facto standard, used by hundreds of independent software vendors worldwide. No other specification is employed on as many hardware platforms, operating systems and applications as PDF. PDF is incorporated into a number of ISO standards, and Adobe encourages developers, independent software vendors and publishers to support and embrace it. While the specification is available publicly, customers expect Adobe to ensure that the format does not become fragmented and that competing implementations of PDF do not undermine what customers have come to expect in terms of reliable viewing and printing of PDF documents across platforms and browsers.

Microsoft has demonstrated a practice of using its monopoly power to undermine cross platform technologies and constrain innovation that threatens its monopolies. Microsoft’s approach has been to “embrace and extend” standards that do not come from Microsoft. Adobe’s concern is that Microsoft will fragment and possibly degrade existing and established standards, including PDF, while using its monopoly power to introduce Microsoft-controlled alternatives – such as XPS. The long-term impact of this kind of behavior is that consumers are ultimately left with fewer choices.

In response, Microsoft released a press release entitled Statement from Microsoft Concerning Adobe which is excerpted below

Microsoft welcomes Adobe’s reaffirmation that PDF is an open standard, made available for free, without restrictions, without royalties, to anyone who cares to use it. Adobe’s frequently stated commitment to PDF as an open standard was the basis upon which Microsoft decided to include support for saving documents in the PDF format in 2007 Microsoft Office system. Customers running Office have clearly said it would be helpful to be able to easily save documents as PDF files. Microsoft expected that Adobe would applaud, rather than object to, our support for the PDF standard in the most widely used productivity software product.

Adobe has now expressed concern that Microsoft would one day “extend” the PDF specifications. Microsoft has not extended the PDF specifications and has no interest in doing so. Our only interest is to support our customers by making it easy to export Office documents in the standard PDF format so they can be viewed in Adobe Reader and printed. Adobe is a participant in the 2007 Office system beta program and can see for itself that Microsoft has not extended the PDF specifications.

To our knowledge Adobe has not imposed limits on how third parties support the PDF format or differentiate their products through extensions. Nevertheless, Microsoft wishes to confirm that it will not extend the PDF specifications published by Adobe. Microsoft hopes that, with this assurance, Adobe will withdraw its objection to including support for PDF in 2007 Office system.

It looks like the ball is now in Adobe's court. I suspect that there is an additional [unstated] concern that the inclusion of PDF support in Microsoft Office reduces the attractiveness of Adobe's suite of products for generating PDF. It would be probably be hard to take that to court, although I've seen  cases which I thought had less merit go against Microsoft. It'll be interesting to see what the response from Adobe will be on this one.


 

The Google Blogoscoped blog has an entry entitled An Inside View From a Google Employee summarizes a very in-depth Something Awful discussion board interview with a developer who claims to work at Google and is currently responsible for the Google Calculator project as his current 20% project. Excerpts from the summary are listed below

  • "Nobody keeps track of 20% time with any care whatsoever. It’s assumed that, if a deadline is pressing on your main project, you’ll work on that. If your main project constantly has looming deadlines, it’s time to talk to your manager or your tech lead and tell them that they’re pushing too hard.” Zorba adds that management understands that a programmer can’t be pushed over limits for more than a week at a time.

    And: “At Google, the managers and tech leads assume that programmers can manage their own time. If a programmer can’t manage their own time they’re probably not a good fit at Google anyway."

  • ZorbaTHut says that Google is mostly C++, Java, and Python (or so he’s been told).
  • On how Google goes about staffing a Test Engineer position, Zorba replies: "I don’t know what other teams are like, but on my team everyone owns their own tests and handles their own quality."

  • Zorba: "[W]e have one monolithic source control system across the entire company. This lets us link in handy libraries from other projects, and is honestly one of the coolest things about working here – if there’s something common you want, chances are good it’s already been written."

  • ZorbaTHut tells us he was assigned on Google Desktop first but didn’t like that much, so he was allowed to switch to working on Google Video. "I actually worked on some neat stuff on Google Video, all of which got cancelled before release. I unfortunately can’t tell you what it was."
  • On what kind of info Google employees are allowed to share:
    "If we haven’t announced it publicly, and it’s a project or a coming feature or anything more financially interesting than ’what color are your carpets’, don’t talk about it. End of story".

  • "The company’s structure, at least for engineers, is amazingly flat."
    Zorba says the Google hierarchy is just five levels: Programmer - Tech lead - Manager - Department lead - Larry/Sergey/Eric. Google just assumes their workers are competent, Zorba adds.

A lot of this jibes with stuff I've heard from about working at Google from Google employees or second hand from friends of employees. Thus I assume the interviewee is legit and is either an employee or someone with a contact inside the company. If he is an employee, he's probably going to get fired for this if Google's past actions are any indication.


 

The has a blog post entitled Is Meta Better? where he writes

Let me start with Digg. It is a proven model.  It works. When we funded delicious, Digg was about half the users and traffic of delicious. They are not the same service, but the delicious popular page and Digg are both places you can go see what's "hot" on the Internet. Since that time, Digg has grown to 1.3mm unique visitors a month compared to only 350k for delicious. These are Comscore Media Metrix numbers. Clearly Digg has a more mainstream user experience. Further, Digg's audience has doubled in the past three months, so it's growing fast.

But let's put this in perspective. Digg's reach (UVs) puts it 735th on Media Metrix' top 1000 sites list, between upromise and taleo. So in the grand scheme of things Digg isn't necessarily the next big thing on the Internet.

I find it surprising that even with the clout of Yahoo! behind it, del.icio.us sees such a relatively small amount of unique users a month. I guess this goes back to the 53,651 meme. I often see people at work using del.icio.us and Digg as examples that a feature or idea is sound but it is clear that these services aren't really popular enough to draw such conclusions.

By the way, given that RSS Bandit has seen over 250,000 downloads this year it looks like we'll soon be eligible to be purchased by Yahoo!. If so, I promise we'll be cheaper than del.icio.us . ;)

PS: Before I get any weird mail, the last comment is a joke. RSS Bandit is BSD licensed so you don't need to pay me us money to do anything you like with it.


 

I saw two related posts on TechCrunch this morning that resonated with some of the issues I face in my day job. The post entitled MySpace Nukes SingleStat.us states

Well, so much for the SingleStat.us experiment that allowed people to find out when someone on MySpace changes their relationship status. MySpace’s friendly lawyers sent a cease and desist letter to David Weekly, the engineer who built SingleStat.us, demanding he take the site down and claiming that his "activities are causing and will continue to cause MySpace substantial and irreparable harm."
...
Something tells me that MySpace might be exaggerating just a bit when they say SingleStat.us is causing them "undue server burden". And if I’ve learned anything from covering companies, it’s that sending a cease and desist letter to a small, one-man startup is generally not going to work out the way you planned.

there is also another post entitled Why is Flickr afraid of Zooomr? which states

Flickr says that users own the the images and tags we enter into their system. Apparently that doesn’t mean they have to make it easy for us to take what we own elsewhere.

When Kristopher Tate, the founder of the feature-rich startup photosharing site Zooomr (see prior coverage), asked Flickr earlier this month for access to their Commercial API, Flickr’s response by email was that "we choose not to support use of the API for sites that are a straight alternative to Flickr." Flickr founder Stewart Butterfield posted to a Flickr forum on Wednesday saying that when it comes to direct competitors like Zooomr, "why should we burn bandwidth and CPU cycles sending stuff directly to their servers?"

The concerns brought up by the MySpace and Flickr folks are concerns that I've had to deal with as our various services such as MSN Spaces and MSN Messenger have grown and we've considered ways to open up these services to third parties via APIs. First of all, despite what Mike Arrington thinks there definitely is a concern when a service shows up whose entire business model is based on polling your website and screenscraping the HTML several times a day. There is a definitley a valid concern if a cottage industry of services springs up around polling MySpace and crawling the various social networks and profiles on the site. Given that there are already necessary evils one has to deal with like search engines and RSS readers, no one who build a successful online service wants to increase the number of web crawlers hitting their site unless absolutely necessary.

However if there are enough of these kinds of requests, it may point to an oppostunity for the website to provide an API. This often ends up building an ecosystem of services and applications aroung the website which ends up benefitting everyone. On the flip side, I often wonder about the balance between making it easy for users to get their data and making it easy for competitors to poach our users. Flickr makes it easy for users to get their data from the service but is hesitant about making it easy for competitors [which includes big players like Google not just startups like Zooomr] from simply bulk copying user information from their service. I think it's a fine line and I've had debates with folks like Mike Torres about where exactly the line is crossed.

In both instances above, the issue isn't the specific incident but the aggregate effect of allowing the behavior to continue.


 

June 15, 2006
@ 07:38 PM

Mike Arrington has a blog post entitled AOL-Netscape Launches Massive "Digg Killer" where he writes

On Thursday, AOL’s Netscape property will no longer be just another portal - it’s being converted into a Digg-killer. I was briefed on the new site by Jason Calacanis last week. As of tonight, he owns the Netscape property at AOL. The new site will run at beta.netscape.com for now, converting over to the main Netscape.com property soon.

It’s not exactly a Digg clone (home page screenshot here). Submitted stories are voted on in much the same way, and the more votes a story gets the higher it appears in a category home page or on Netscape.com itself. However, the top few spots in each category and on the home page are determined by an "anchor" - essentially an editor choosing from stories moving up the ranks.

There's also some good commentary on this in Greg Linden's post Netscape's News Scrape and Danny Sullivan in his post Netscape Aims To Be Digg 2.0, Slashdot 3.0 With Community News Model. Using the wisdom of the crowds to improve news sites is an good idea and one that has worked for Slashdot and Digg as Danny Sullivan points out. However I agree with Greg Linden that the holy grail is truly personalized news although the Slashdot/Digg/Netscape approach is a decent middle ground.

I was going to send a mail at work about this and encourage Microsoft to do the same until I realized I didn't even know which division of folks to send the mail to. If you read any random Microsoft press release you'll see the following excerpt

MSN is a world leader in delivering compelling programmed content experiences to consumers and online advertising opportunities to businesses worldwide. Windows Live, a new set of personal Internet services and software, is designed to bring together in one place all the relationships, information and interests people care about most

So MSN is for programmed content like your traditional news site and Windows Live is for personalized content. So which of these two divisions would be interested in a half-way approach like Slashdot or Digg? I literally have no idea and I work at Microsoft. So I cancelled the mail and blogged this instead. If I'm confused by this arbitrary distinction between MSN and Windows Live, I can only imagine how much our customers get it. :)


 

Tim O'Reilly ran a series of blog posts a few months ago on the O'Reilly Radar blog entitled "Database War Stories" where he had various folks from Web companies talk about their issues scaling databases. I thought the series of posts had an interesting collection of anecdotes and thus I'm posting this here so I have a handy link to the posts as well as the highlights from each entry.

  1. Web 2.0 and Databases Part 1: Second Life: Like everybody else, we started with One Database All Hail The Central Database, and have subsequently been forced into clustering. However, we've eschewed any of the general purpose cluster technologies (mysql cluster, various replication schemes) in favor of explicit data partitioning. So, we still have a central db that keeps track of where to find what data (per-user, for instance), and N additional dbs that do the heavy lifting. Our feeling is that this is ultimately far more scalable than black-box clustering.

  2. Database War Stories #2: bloglines and memeorandum: Bloglines has several data stores, only a couple of which are managed by "traditional" database tools (which in our case is Sleepycat). User information, including email address, password, and subscription data, is stored in one database. Feed information, including the name of the feed, description of the feed, and the various URLs associated with feed, are stored in another database. The vast majority of data within Bloglines however, the 1.4 billion blog posts we've archived since we went on-line, are stored in a data storage system that we wrote ourselves. This system is based on flat files that are replicated across multiple machines, somewhat like the system outlined in the Google File System paper, but much more specific to just our application. To round things out, we make extensive use of memcached to try to keep as much data in memory as possible to keep performance as snappy as possible.

  3. Database War Stories #3: Flickr: tags are an interesting one. lots of the 'web 2.0' feature set doesn't fit well with traditional normalised db schema design. denormalization (or heavy caching) is the only way to generate a tag cloud in milliseconds for hundereds of millions of tags. you can cache stuff that's slow to generate, but if it's so expensive to generate that you can't ever regenerate that view without pegging a whole database server then it's not going to work (or you need dedicated servers to generate those views - some of our data views are calculated offline by dedicated processing clusters which save the results into mysql).

  4. Database War Stories #4: NASA World Wind: Flat files are used for quick response on the client side, while on the server side, SQL databases store both imagery (and soon to come, vector files.) However, he admits that "using file stores, especially when a large number of files are present (millions) has proven to be fairly inconsistent across multiple OS and hardware platforms."

  5. Database War Stories #5: craigslist: databases are good at doing some of the heavy lifting, go sort this, give me some of that, but if your database gets hot you are in a world of trouble so make sure can cache stuff up front. Protect your db!

    you can only go so deep with master -> slave configuration at some point you're gonna need to break your data over several clusters. Craigslist will do this with our classified data sometime this year.

    Do Not expect FullText indexing to work on a very large table.

  6. Database War Stories #6: O'Reilly Research: The lessons:

    • the need to pay attention to how data is organized to address performance issues, to make the data understandable, to make queries reliable (i.e., getting consistent results), and to identify data quality issues.
    • when you have a lot of data, partitioning, usually by time, can make the data usable. Be thoughtful about your partitions; you may find its best to make asymmetrical partitions that reflect how users most access the data. Also, if you don't write automated scripts to maintain your partitions, performance can deteriorate over time.
  7. Database War Stories #7: Google File System and BigTable: Jeff wrote back briefly about BigTable: "Interesting discussion. I don't have much to add. I've been working with a number of other people here at Google on building a large-scale storage system for structured and semi-structured data called BigTable. It's designed to scale to hundreds or thousands of machines, and to make it easy to add more machines the system and automatically start taking advantage of those resources without any reconfiguration. We don't have anything published about it yet, but there's a public talk about BigTable that I gave at University of Washington last November available on the web (try some searches for bigtable or view the talk)."

  8. Database War Stories #8: Findory and Amazon: On Findory, our traffic and crawl is much smaller than sites like Bloglines, but, even at our size, the system needs to be carefully architected to be able to rapidly serve up fully personalized pages for each user that change immediately after each new article is read. Our read-only databases are flat files -- Berkeley DB to be specific -- and are replicated out using our own replication management tools to our webservers. This strategy gives us extremely fast access from the local filesystem. We make thousands of random accesses to this read-only data on each page serve; Berkeley DB offers the performance necessary to be able to still serve our personalized pages rapidly under this load. Our much smaller read-write data set, which includes information like each user's reading history, is stored in MySQL. MySQL MyISAM works very well for this type of non-critical data since speed is the primary concern and more sophisticated transactional support is not important.

  9. Database War Stories #9 (finis): Brian Aker of MySQL Responds: Brian Aker of MySQL sent me a few email comments about this whole "war stories" thread, which I reproduce here. Highlight -- he says: "Reading through the comments you got on your blog entry, these users are hitting on the same design patterns. There are very common design patterns for how to scale a database, and few sites really turn out to be all that original. Everyone arrives at certain truths, flat files with multiple dimensions don't scale, you will need to partition your data in some manner, and in the end caching is a requirement."

    I agree about the common design patterns, but I didn't hear that flat files don't scale. What I heard is that some very big sites are saying that traditional databases don't scale, and that the evolution isn't from flat files to SQL databases, but from flat files to sophisticated custom file systems. Brian acknowledges that SQL vendors haven't solved the problem, but doesn't seem to think that anyone else has either.

I found most of the stories to be interesting especially the one from the Flickr folks. Based on some early thinking I did around tagging related scenarios for MSN Spaces I'd long since assumed that you'd have to throw out everything you learned in database class at school to build anything truly performant. It's good to see that confirmed by more experienced folks.

I'd have loved to share some of the data we have around the storage infrastructure that handles over 2.5 billion photos for MSN Spaces and over 400 million contact lists with over 8 billion contacts for Hotmail and MSN Messenger. Too bad the series is over. Of course, I probably wouldn't have gotten the OK from PR to share the info anyway. :)


 

Joe Wilcox of Jupiter Research has a blog post entitled Google My Spreadsheet where he talks about Google's recently announced Web-based spreadsheet application. He writes

So, Google is doing a spreadsheet. On the Web. With collaboration. And presumably--if and when released--Google Spreadsheet will be available for free. I predict there will be crisis meetings at Microsoft today. I totally agree with Colleague David Card that "Google is just playing with Microsoft's (hive) mind. Scaring the troops. Sleight-of-handing the managers."

Perhaps the real Google competitive threat isn't any product or products, but the information vendor's ability to rustle Microsoft corporate paranoia. To get Microsoft chasing Google phantoms, and in the process get Microsoft's corporate mind off its core business. News media will be gaga with Google competing with Microsoft stories--two juggernauts set to collide. Yeah, right. I'd be shocked if Google ever released a Web browser, operating system or desktop productivity suite. Those markets aren't core to Google's business, contrary to speculation among news sites and bloggers.

As for the spreadsheet, which isn't really available beyond select testers, what's it really going to do? David is right, "Consumers don't use spreadsheets. No thinking IT manager would sign off on replacing Excel with a Web-based spreadsheet." Google's target market, if there really is one, appears to be casual consumer and small business users of spreadsheets--people making lists. OK, that competes with Microsoft how? So soccer mom or jill high schooler can work together with other people from the same based Web-based spreadsheet. Microsoft shouldn't really sweat that, although Microsoft might want to worry about what Google might do with extending search's utility.

I agree 100% with Joe Wilcox's analysis here. This seems more like a move by Google to punk Microsoft into diverting focus from core efforts than a product category that is well thought out. I thought at the recent Google Press Day, Eric Schmidt mentioned that they have not been doing a good job of following the 70/20/10 principle (70 percent focus on core business, 20 percent on related businesses and 10 percent on new businesses).

If I was a Google investor, I'd be worried about the fact that their search engine relevance is detoriorating (Google Search for "msdn system.string" doesn't find this page in the top 10 results) and they are wasting resources fragmenting their focus in so many myriad ways. As a competitor, it makes me smile. ;) 

Update: From the Google blog post entitled It's nice to share it looks like this is an offerings from the creators of XL2Web who were acquired by Google a year ago. Another example of innovation by acquisition by Google? Interesting.

 

June 2, 2006
@ 03:41 PM

Mark Cuban has blog post entitled Why I think ClickFraud is far greater than imagined where he lays out some anecdotal reasons why he thinks the sky is falling for Pay-Per-Click (PPC) advertising due to click fraud. He writes

Now i have no idea how much money is being lost to click fraud. All i know is that when the black hat hackers see easy money, they take it. I also know that they are greedy and a jealous bunch. The more they see the more they take, so you can pretty well bet that the amount of click fraud is going up by the minute.

And no amount of IP repetition algorithms are going to stop them.

Again, this is all opinion. No investigative reporting going on. Yet.

I have no hard numbers on how much is being "lost" to click fraud but there are a number of reasons why I'm skeptical about how much attention people pay to scare mongering about click fraud and it's effects on companies like Google when 'the market corrects'.

Reason number one is that despite how much people may complain about PPC advertising, it works a lot better than the alternatives. Mark Cuban actually just wrote a blog post a few days ago entitled A quick letter to the Newspaper and Magazine Industries where he complains about how expensive traditional advertising is compared to the returns. If you were trying to drive traffic to your website and had a million dollars to spend would you spend it on newspaper ads, television ads or AdSense/Adwords ads? As long as the return on investment (ROI) for PPC ads is higher than other forms of advertising, I suspect advertisers will consider the money lost to click fraud as acceptable losses. This is no different from retail stores which have to accept a certain amount of loss from shop lifting and customer returns yet still remain profitable.

Another reason I'm skeptical about fear mongering around click fraud is that this isn't the first time technology has made it easy to subvert a market yet those markets still exist. My employer (Microsoft) has built one of the most profitable businesses in the world selling products that can be copied by anyone with a CD burner and a computer. In college, I don't think I knew many people who paid for their copies of Microsoft Office yet that business still manages to bring in billions of dollars in profit a year. Then there are other more recent markets like ring tones. This has emerged as a multi-billion dollar industry in the past few years even though it is quite possible for people to get ringtones for free on their cell phones without much hassle. And then there's the iTunes Music Store

I'd be unsurprised if there is a larger than commonly assumed amount of click fraud going on out there. No more surprised than I'd be to find out that software/music piracy rings exist, insurance fraud is about 15 %- 30% of insurance claims or that shoplifting costs retailers billions of dollars every year. I don't see people ringing the death knell of WalMart or Safeco because of these grim statistics. So the next time you see someone claiming that the sky is falling because of click fraud, remember that fraud exists in every aspect of human commerce and the Web isn't going to change that. 


 

Greg Linden has a blog post entitled Early Amazon: Shopping Cart Recommendations where he writes

I loved the idea of making recommendations based on the items in your Amazon shopping cart. Add a couple things, see what pops up. Add a couple more, see what changes. The idea of recommending items at checkout is nothing new. Grocery stories put candy and other impulse buys in the checkout lanes. Hardware stores put small tools and gadgets near the register. But here we had an opportunity to personalize impulse buys. It is as if the rack near the checkout lane peered into your grocery cart and magically rearranged the candy based on what you are buying.
...
I hacked up a prototype. On a test site, I modified the Amazon.com shopping cart page to recommend other items you might enjoy adding to your cart. Looked pretty good to me. I started showing it around. While the reaction was positive, there was some concern. In particular, a marketing senior vice-president was dead set against it. His main objection was that it might distract people away from checking out -- it is true that it is much easier and more common to see customers abandon their cart at the register in online retail -- and he rallied others to his cause.

At this point, I was told I was forbidden to work on this any further. I was told Amazon was not ready to launch this feature. It should have stopped there. Instead, I prepared the feature for an online test. I believed in shopping cart recommendations. I wanted to measure the sales impact.

I heard the SVP was angry when he discovered I was pushing out a test. But, even for top executives, it was hard to block a test. Measurement is good. The only good argument against testing would be that the negative impact might be so severe that Amazon couldn't afford it, a difficult claim to make. The test rolled out
...
The results were clear. Not only did it win, but the feature won by such a wide margin that not having it live was costing Amazon a noticeable chunk of change. With new urgency, shopping cart recommendations launched.

On the O'Reilly Radar site, Marc Hedlund points to this post as an example company leadership that encourages employees to not only come up with innovations but go head-to-head with management to get them out to customers.

It's interesting how different people look at the same story. When I originally read the story, what jumped out to me was that Amazon must have a great A/B testing framework which allows them to measure such tweaks to the user experience of their site so accurately. Coincidentally, I just had lunch with one of the folks at work who is building a generic A/B testing framework for Windows Live, MSN and third party developers; the Microsoft Experimentation Platform.

I wonder how many web companies have the infrastructure to test and measure the kind of change Greg prototyped at Amazon on their website today? Probably not a lot. We should fix that.


 

I found the following comments by Om Malik and Mike Arrington to be quite telling.

In his blog post entitled The Myth, Reality & Future of Web 2.0 Om Malik writes

The Myth of Web 2.0 is the investment opportunities. The reality of Web 2.0 is too little original thinking. Web 2.0, simply put, is a set of technologies and a new kind of thinking, which companies like Yahoo, Google, Microsoft and AOL are incorporating in their products. That’s the reality and the future of Web 2.0.

In the blog post entitled AOL To Release YouTube Clone Mike Arrington writes

Prepare for the launch of AOL UnCut (currently in open beta), a near perfect clone of YouTube...This is right on the heels of the launch of AIM Pages, which is directly targeting Myspace and other social networks...I am seeing an increasing trend of the big guys simply copying what successful startups are doing. AOL with this product and AIM Spaces. Google with Google Notepad and a flurry of other projects, etc. The only large company that is even experimenting with unproven concepts at this point is Microsoft with its various Live.com ideas. I’d like to see more experimenting at the big company level.

I guess the criticism has now grown from 'building a new Windows app is just doing research for Microsoft' to 'building a new Web application is just doing research for Google/Yahoo/AOL/Microsoft'. The more things change, the more they stay the same.

On the positive side, it is good to see Microsoft being called innovative in comparison to Google by a technology pundit.


 

May 10, 2006
@ 03:22 PM

I stumbled on a post entitled Using Gmail for work from Rakesh Agrawal, President and CEO of Snapstream, about switching from Microsoft Outlook and Exchange to GMail for his work email. He writes

After going through my fourth or fifth Microsoft Exchange crash and countless Outlook problems (after 3 years!), I decided that I had had enough. For a little over a month, I've been using Gmail as my primary client for e-mail -- for work e-mail, for personal e-mail and everything in between. So far I love it, though I've discovered that there are also a few things that make it undesirable. Read on for the details...

The Benefits

For me, there have been countless benefits of running email in the network cloud and not on a local client on my PC. Let me count the ways...

  • - No more crash-prone exchange server or outlook clients:
  • - Access from literally any computing device with Internet access:
  • - Gmail Mobile (m.gmail.com):
  • - reliable and effective search:
  • - spam filtering is really, really good
  • - filtering is fast, simple (just like search):

The Downsides

  • - E-mail accounts functionality could be a lot better:
  • - m.gmail.com doesn't support "accounts":
  • - No offline access:
  • - It's not a local application:
  • - Space limitations:
  • - Formatting limitations:
  • - Occasional hiccups of service:

Most of his complaints seem to be already fixed in existing versions of Exchange. I often access my Outlook mail using the Outlook Web Access functionality that has been a part of Exchange for years [and is the reason XMLHttpRequest exists in the first place]. In fact VPNing has been such a hassle for me at home that I often resort to a combination of OWA and the RPC over HTTP feature of Outlook 2003. I get mobile access fine from my Audiovox SMT 5600 phone which syncs both my mail and calendar which has quite literally changed my life. Of course, it does mean that I check work email when I shouldn't such as when I'm at airports supposedly on vacation.For search, I use Windows Desktop Search which works great. However it is a fair point that there is no decent equivalent if accessing mail from the Web or a mobile phone. 

I found this post via Nathan Weinberg who addresses the offline problem in his commentary on Rakesh's post by adding

So, how can Google address his caveats? The biggest seems to be an offline version of Gmail, something Google has not indicated it is developing. I think the Windows Live Mail Desktop solution is the way to go, designing a light, yet very well featured desktop email client that is clearly designed to work in conjunction with web-based email, not as a replacement.

I totally agree with that approach because it marries the best of the Web and the desktop instead of one trying to replace the other. This topic points out the kind of interesting tensions that I believe exist at Microsoft today. The Exchange team should look at posts like Rajesh and wonder how to stay competitive with Web-based solutions but I suspect they also wonder if products like Windows Live Mail and Office Live aren't cannibalizing their business. Ideally, we should offer a continuoum of products which should include Web-based offerings for companies that don't want the hassle/cost of managing their own mail servers to enterprise offerings for the bigcos who can't fathom why any business wouldn't want all their mail to be processed and stored in-house. This seems to be the tack that we are taking today, which is great, but it definitely leads to interesting conversations.

One size doesn't fit all. Always remember that. 

NOTE: I inserted the link to the Windows Live Mail Desktop team's blog to Nathan's post.


 

Nathan Weinberg who writes the Inside Google blog has a blog entry entitled Screw YouTube where he writes

Miel’s quit YouTube. Considering he introduced me to the service, which I began to love, contribute to, and trumpet as the next great success story, you’d think I’d be surprised. Not even a little.

See, I got kicked off YouTube over two months ago. The reason? Contributing to the success of their service. I uploaded a good number of videos to YouTube, almost none of which I owned the copyright to, all of which I got from other sources on the internet. My first video, the “banned” Xbox 360 ad, was for a time the second most watched video on YouTube, with close to two million viewings.

On February 24, I received two emails, detailing how a video I had posted, a Saturday Night Live sketch in which President Bush asks a Santa Dick Cheney for an Xbox 360, had been rejected due to a third party notification of infringement.

Anyway, I’m done with YouTube, almost. It is clear they have no interest in preserving a digital archive of video content for the future, and that I cannot rely on them for posterity...I do have one thing left to do: Ruin YouTube. Since it is so easy to get someone kicked from YouTube, I am going to launch an assault on the service...Every day, I will destroy at least one account. I will only target those with copyright infringing content. When I am done, the only popular videos on YouTube will be those with zero commercial value. We will see how well the service does without the Daily Show and South Park entire episodes that are its real bread and butter.

I am extremely surprised at such a vindictive and destructive response by Nathan Weinberg to what I see as a reasonable act on the part of YouTube. From my perspective, YouTube is a video sharing service which is likely to make a bunch of money [via ads] serving content that doesn't belong to them. Even if it wasn't illegal I personally think this is unethical. YouTube shouldn't be making money off of TV shows like Daily Show and South Park instead of the creators and/or owners of the copyright on these shows. I find it commendable that the folks at YouTube are trying to make sure they don't become a leech on the system and instead are a way to provide an avenue for long tail content which you cannot find via traditional broadcast media. Of course, this is just common sense on the part of the YouTube folks since they want to avoid the same mistakes made by Napster.

On the flip side I can't help but remember Danah Boyd's excellent paper, Friendster lost steam. Is MySpace just a fad? which argues that one of the reasons that Friendster lost steam is that it failed to recognize and bow down to the wishes of core members of its user base. This lead to alienation and outright hostility from users who were once major users and proponents of the service. Reading Nathan Weinberg's post, I wonder of YouTube is going down the same path. 


 

Danny Sullivan of the Search Engine Watch journal has a blog post entitled Google Worried About Microsoft's Browser Advantage? What Advantage? where he writes

I am nauseatingly exhausted by idea that Microsoft will conjure up some magical method of yanking people into its MSN Windows Live Whatever You Want To Call It search service via the Windows operating system or the Internet Explorer browser. Microsoft has failed for years to be successful in this, which is why it's amazing anyone would still believe it.

In the longer version of this post for Search Engine Watch members, I revisit the tired facts in more depth:

  • How search has been integrated into Windows and Internet Explorer since 1996 but failed to help Microsoft.
     
  • How even when MSN Search was made the default choice by 2001, Google still rose in traffic share.
     
  • How putting the search box into the "chrome" of the browser doesn't necessarily mean Microsoft will have a major win this time.
     
  • How search via toolbars still remain the minority of the way searches happen.

Meanwhile, skip past the business aspects. What about the consumer issue of choice? The New York Times writes of Google's preferred solution:

The best way to handle the search box, Google asserts, would be to give users a choice when they first start up Internet Explorer 7. It says that could be done by asking the user to either type in the name of their favorite search engine or choose from a handful of the most popular services, using a simple drop-down menu next to the search box. The Firefox and Opera browsers come with Google set as the default, but Ms. Mayer said Google would support unfettered choice on those as well.

Sure, I can get behind the "give people a choice from the beginning" idea. But if Google wants Microsoft to do that, then Google should make it happen right now in Firefox, which pretty much is Google's surrogate browser. If this is the best way for a browser to behave, then Google should be putting its weight on Firefox to make it happen. And Google should also ensure it does the same with Dell, where it has a partnership that I believe makes it the default search engine on new Dell computers.

There definitely has been a bunch of interesting commentary on this topic. Check it out on tech.memeorandum.
 

In the blog post entitled Rapleaf to Challenge eBay Feedback Mike Arrington talks about newly formed Rapleaf which aims to build a competitor to eBay's feedback system. This idea shows a lot of insight on the part of the founders. The value that eBay provides to sellers and buyers is primarily the reputation system and not as a venue for auctions. The network effects inherent in eBay's reputation system make it the ultimate kind of lock-in. No power seller or buyer will look at alternatives even if they are free (like Yahoo! Auctions) because they don't want to start from scratch with the reputation they've built or trust trading with people whose reputations haven't been built. However it isn't a slam dunk that Rapleaf will be a successful idea. 

In his post entitled Rapleaf's Fatal Flaws Ian McAllister of Windows Live Shopping writes

Flaw #1 - Transaction Unaware
Rapleaf is not in the middle of transactions. They have no way to determine if a transaction between two parties actually took place. Co-founder Auren Hoffman claims that their sophisticated human and machine-based fraud detection will be able to detect fraud but to me this seems like complete hand-waving...The success of eBay's feedback system rests completely on the fact that they attach feedback only to completed transactions where eBay collects money via commission.
 
Flaw #2 - Cold Start
Every new startup has a cold start problem and must build users, customers, partners, etc. from the ground up but Rapleaf has the mother of all cold start problems. The post mentioned nothing about how they plan to build mindshare in the market and I think they'll be dead in the water if they expect users to start going to www.rapleaf.com in droves all of a sudden and being keen to trust one of the 342 Rapleaf trusted sellers based on 2 items of feedback not attached to any verified transaction.

Flaw #2 was something I'd considered but Flaw #1 didn't even occur to me. Now that I consider it, I can't see how they can be successful as a competitor of companies like eBay since they aren't part of the transaction. It would seem to make more sense for them to be a partner of eBay except that there is no incentive for eBay to partner with them and thus provide an avenue out of the lock-in of eBay's feedback system.

Does this mean Rapleaf is DOA?