Every product or business leader understands the importance of having the right strategy. This includes being deeply aware of your competitive landscape, understanding your customers and how they use your product and ensuring you have the right plan to continue to make your customers happy while moving your business forward. However few take into account the importance of the alignment of your current strategy and your organization’s culture. This is especially important as strategies may evolve time as the marketplace changes while organizational cultures are fairly stagnant.

In today’s technology landscape the only constant is change. Companies that are impervious one day are dwarfed by upstarts seemingly the next day. Whether it is Uber leading to the bankruptcy of Yellow Cab in San Francisco, Netflix eclipsing Blockbuster or Apple’s iPhone leading to the decline Blackberry there are multiple examples in recent memory of established products that seemed to be a permanent fixture of the world that were bested by others that gave customers a better experience. In all of these cases, there are examples of the incumbent struggling and failing to adapt to the new world. Yellow Cab companies created mobile apps like YoTaxi and Blockbuster tried offering DVDs by mail and Blackberry eventually offered touch screen phones like the Storm. A key problem for all of these disrupted companies was that they tried to switch strategies but could not surmount the handicaps of their organizational cultures and fundamental approach to doing business.

Culture Shock: How Blackberry Failed to Respond to the iPhone

There is a great overview of the downfall of Blackberry in the Globe & Mail article titled Inside the fall of BlackBerry: How the smartphone inventor failed to adapt which is excerpted below

RIM executives figured they had time to reinvent the company. For years they had successfully fended off a host of challengers. Apple’s aggressive negotiating tactics had alienated many carriers, and the iPhone didn’t seem like a threat to RIM’s most loyal base of customers – businesses and governments. They would sustain RIM while it fixed its technology issues.

But smartphone users were rapidly shifting their focus to software applications, rather than choosing devices based solely on hardware. RIM found it difficult to make the transition, said Neeraj Monga, director of research with Veritas Investment Research Corp. The company’s engineering culture had served it well when it delivered efficient, low-power devices to enterprise customers. But features that suited corporate chief information officers weren’t what appealed to the general public.

“The problem wasn’t that we stopped listening to customers,” said one former RIM insider. “We believed we knew better what customers needed long term than they did. Consumers would say, ‘I want a faster browser.’ We might say, ‘You might think you want a faster browser, but you don’t want to pay overage on your bill.’ ‘Well, I want a super big very responsive touchscreen.’ ‘Well, you might think you want that, but you don’t want your phone to die at 2 p.m.’ “We would say, ‘We know better, and they’ll eventually figure it out.’ ”

In reading the article it is clear that at that the Blackberry leadership decided that it was important for them strategically to address the rise of the iPhone. However as the unnamed insider makes clear above, their organizational culture was simply not set up to make the mental shift to view their product and needs customers differently after the iPhone launched.

There is a great quote from Upton Sinclair “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

Blackberry considered its primary customers to be carriers and businesses since they were the ones paying for devices and services. Carriers did not want phones with capable browsers because they didn’t want high cellular data usage to overwhelm their network while businesses who bought phones for their employees did not want to have said employees goofing off in apps. This is why there was a strong disincentive to not listen to what users of their phones were saying when asked what they actually wanted. On the other hand, Apple approached the problem by thinking about how to provide the very best experience to users of their phones with the confidence that people would happily pay a premium for it.

Blockbuster faced a similar challenge with Netflix. A key part of Blockbuster’s business was charging people late fees. This meant that Blockbuster was literally making money off of the inconvenience of their product experience. This meant that as an organization competing with Netflix on customer experience would have meant attacking one of their cash cows and there was a strong disincentive not to do that.

These are just two of many examples that highlight the point that when your strategy changes then your entire organizational culture will have to change as well. Your organizational culture is defined by what positive behaviors you encourage and what negative behaviors you tolerate. Blackberry couldn’t compete with Apple when teams were still motivated & rewarded for keeping corporate CIOs happy and there was no way Blockbuster could compete with Netflix when they fundamentally saw themselves as a classic retail video rental store and ignored the power of online experiences.

Cultural Appropriation: How Google’s Android Project Responded to the iPhone

There are examples of other companies adjusting their strategies and engineering culture when faced with dramatic change in their industry. My favorite is taken from this excerpt from the article The Day Google Had to 'Start Over' on Android which is excerpted below

In 2005, on Google’s sprawling, college-like campus, the most secret and ambitious of many, many teams was Google’s own smartphone effort—the Android project. Tucked in a first-floor corner of Google’s Building 44, surrounded by Google ad reps, its four dozen engineers thought that they were on track to deliver a revolutionary device that would change the mobile phone industry forever.

As a consumer I was blown away. I wanted one immediately. But as a Google engineer, I thought ‘We’re going to have to start over.’

By January 2007, they’d all worked sixty-to-eighty-hour weeks for fifteen months—some for more than two years—writing and testing code, negotiating soft­ware licenses, and flying all over the world to find the right parts, suppliers, and manufacturers. They had been working with proto­types for six months and had planned a launch by the end of the year . . . until Jobs took the stage to unveil the iPhone.

Chris DeSalvo’s reaction to the iPhone was immediate and visceral. “As a consumer I was blown away. I wanted one immediately. But as a Google engineer, I thought ‘We’re going to have to start over.’”

The way Google reacted to the iPhone is quite telling especially when contrasted with the Blackberry story. The Android team spent two years working 60 – 80 hours a week to deliver a new phone operating system but once they saw that Apple had raised the bar they decided they needed to go back to the drawing board. It is quite stark to compare what Android prototypes looked like before the launch of the iPhone and what consumers associate with Android after it launched.

Android Before the iPhone launched	Android After the iPhone launched

It would have been really easy for the team working on Android to continue down their current path at the time. They’d already spent 2 years working hard on a Blackberry-style phone and knew that was already a proven lucrative market. However with a culture focused on customer experience, they realized that Apple’s path and not Blackberry’s was the future and they reimagined their product.

Facebook: An Upstart Becomes an Incumbent and Avoids a Culture of Complacency.

Facebook dethroning MySpace as the top social network in the US and the going on to displace the majority of the regionally popular social networks from around the world like Bebo, Orkut and Mixi is a good example of an upstart dethroning incumbents. On the other hand what has been even more impressive is how Facebook evolved as smartphones became more popular.

A few years ago it was commonplace to read articles like Here's Why Google and Facebook Might Completely Disappear in the Next 5 Years which argued the following

Facebook is also probably facing a tough road ahead as this shift to mobile happens.  As Hamish McKenzie said last week, “I suspect that Facebook will try to address that issue [of the shift to mobile] by breaking up its various features into separate apps or HTML5 sites: one for messaging, one for the news feed, one for photos, and, perhaps, one for an address book. But that fragments the core product, probably to its detriment.”

Considering how long Facebook dragged its feet to get into mobile in the first place, the data suggests they will be exactly as slow to change as Google was to social.  Does the Instagram acquisition change that? Not really, in my view.  It shows they’re really fearful of being displaced by a mobile upstart.  However, why would bolting on a mobile app to a Web 2.0 platform (and a very good one at that) change any of the underlying dynamics we’re discussing here? I doubt it.

At the time there was significant belief that since Facebook was dominant on the desktop and quite dependent on desktop-based revenue from Facebook games like Farmville for a huge chunk of its revenue, it would act like a typical incumbent and double down on where it was making money while an upstart beat them on mobile. So where the pundits right?

Here’s a report from 3 years after the one above which proclaims

The world's largest social network said nearly 84 percent of the 890 million people who used its service daily did so on a mobile phone. Nearly 86 percent of the 1.39 billion people who accessed Facebook each month did so on a mobile device as well, a new record for the company.

Advertisers followed those numbers. Mobile ads accounted for about 69 percent of the company's $3.85 billion in revenue. Overall, the company's sales jumped nearly 49 percent from the same time a year ago.

Those numbers underscore the Menlo Park, Calif., company's increasing reliance on mobile devices for its business. Much of the technology industry has become fixated on smartphones and tablets, as people throughout the world switch from desktop computers. Investors are now paying more attention to the mobile aspects of Facebook and its competitors. Few companies have successfully navigated the switch to mobile devices as effectively as social networks, like Facebook and Twitter.

In 3 years, Facebook went from a dead man walking due to the rise of the smartphone to becoming the biggest success story on mobile by making the most money and having the most users of any other mobile app. Facebook has done this by having a culture that comes across as almost paranoid when it comes to losing users to competitors which is why they do things like forcing employees to use Android (the most popular mobile OS), copying features like mad from startups as diverse as Medium, Timehop and Twitter, and seemingly overspends to acquire mobile startups like Instagram & WhatsApp even though the usage of their core apps is still growing like crazy.

Other companies, most notably Google, have tried to unseat Facebook from its perch on top of the social networking world but none has been able to match their paranoid frenzied approach of building or buying every interesting thing happening in social media. This aspect of Facebook’s culture and approach to their business has proven hard to copy or defeat.

How This Applies to Your Organization

As mentioned earlier, your organizations culture is defined by the positive behavior you reward and the negative behaviors you tolerate. When you define your strategy you have to also take a hard look at the behaviors you reward within your organization to ensure that the culture you have created aligns with your strategy. My current employer, Microsoft, is in the midst of a strategic redefinition and culture change as I write this and it’s been interesting to see the big and small changes that have occurred. From how sales teams are compensated & how performance reviews across the company have been tweaked to which engineering projects are now approved & celebrated, there have been numerous changes made with the goal of ensuring the company’s culture doesn’t conflict with its new strategy.

So far the stock market has been very receptive to these changes over the past 2 years which is a testament to how well things can go if your new strategy & corporate culture align.

Now Playing: Young Thug – Power

As I write this the latest version of Skype for the iPhone has a 2 star rating as does Swarm by Foursquare. What these apps have in common is that they are both part of bold attempts to redesign a well-known and popular app which are being rejected by its core constituency. A consequence of my time working on Windows 8 is that I now obsess quite a bit about redesigning apps and determining what warning signs indicate that you are either going to greatly please or strongly offend your best users.

When I worked on Windows 8, there were a number of slogans that the team used to ensure the principles behind the work we were doing were understood by all. Some of them such as “content over chrome” were counterproductive in that slavish devotion to them led to ignoring decades of usability research by eschewing affordances and hiding navigation/controls within apps. However there were other principles from the Windows 8 era which I wish app developers took more to heart such as “win as one” which encouraged consistency with the overall platform’s UI model & working with other apps and “change is bad unless it's great” which encouraged respecting the past and only making changes that provided a noticeably better user experience. 

In addition to these two principles, I’ll add one more for app developer to keep in mind whenever the time calls for a redesign; “minimize the impacts of loss aversion”. For those who aren’t familiar with the term, loss aversion (aka the endowment effect) is the tendency for humans to strongly prefer avoiding losses to making gains. What this means for developers is that end users will react more strongly to losing a feature than they would to gaining that same feature. There are numerous studies that show how absurd humans can be in the face of loss aversion no matter how minor. My favorite example is how much people overreact to loss aversion when it comes to grocery shopping as taken from this blog post by Jon Geeting

There was a law set up last month in D.C. (passed unanimously by city council) to place a five-cent tax on paper and plastic bags at grocery stores, pharmacies and other food-service providers. So, basically, if I went shopping, my total came to $35.20, and I needed one bag to put it in, my total would then become $35.25. Similarly, if I needed two bags, my total would become $35.30, and so on — while if I simply bought reusable bags, I would be subject to no tax.

From what I hear from people in D.C., they absolutely hate it. Even though it’s just an extra five cents, they want absolutely nothing to do with it. They really want that nickel. So many people use less bags, bring their own, or just try to balance everything without one on their trip home. Think about how much less waste and pollution there is in D.C. now, because of a measly five-cent fee.

On the flip side, if you told people you’d give them 5 cents for each bag they brought from home they’d laugh in your face. Nobody is going to do an extra bit of work to be paid five cents even though they would do that work to avoid paying 5 cents. That’s loss aversion at work.

To recap, if you are redesigning an app you need to keep these three rules in mind

1. Win as one: Whatever changes you make must feel like a consistent whole both within the app and with the platform your app resides on. Swarm and Foursquare have completely different aesthetics and integrate in a fairly disjointed manner often with no way to easily jump back and forth between both apps. Skype for iPhone is pretty much a Windows Phone app in look and feel complete with pivot controls and cut off title text. This is a very jarring experience compared to everything else on iOS.

2. Change is bad unless it’s great: App developers need to be honest with themselves about whether a redesign is about solving a customer problem in a better way or is part of a corporate strategy. Facebook news feed is an example of a redesign which was actually driven by a need to solve customer problems which is why although it met with a massive user revolt at first, once people used it they loved it and the anger died down. Swarm exists because FourSquare now wants to compete with Yelp and needs to shed its history as a social check-in app which it sees as baggage as it evolves into a social discovery engine of things to do in your city. From an end user perspective, Skype for iPhone’s redesign is about making the app look and feel like a Microsoft metro-style app. Given these primary goals, it is no surprise that end users can tell that solving their problems came in second place as they review these apps.

3. Minimize the impacts of loss aversion: Coupling a redesign with taking away features means people will focus on the missing features instead of whatever benefits you have provided with the redesign. Foursquare took away badges, mayorships, social feed of your friends check-ins and points as part of the split that created Swarm. There are a large number of one star reviews of the Swarm app complaining about these missing features. Skype for iPhone’s initial release took away deleting & editing messages while making others harder to find. Even features that are used once in a blue moon seem mission critical once people find out they are gone. Taking away features will always sting more than the actual value of those features. Taking multiple features away as part of a redesign means any benefits of the redesign will be lost in the ensuing outrage about the missing features.

Now Playing: Rick Ross – The Devil is a Lie (featuring Jay-Z)

Chris Dixon has a fairly eloquent blog post where he talks about the decline of the mobile web. He cites the following chart

and talks about what it means for the future of innovation if apps which tend to be distributed from app stores managed by corporate gate keepers continue to dominate the web as the primary way people connect on the Internet.

Using HTTP Doesn’t Make Something Part of the Web

In response to Chris Dixon’s post I’ve seen a fallacy repeated a number of times. The most visible instance of this fallacy is John Gruber’s Rethinking What We Mean by ‘Mobile Web’ where he writes

I think Dixon has it all wrong. We shouldn’t think of the “web” as only what renders inside a web browser. The web is HTTP, and the open Internet. What exactly are people doing with these mobile apps? Largely, using the same services, which, on the desktop, they use in a web browser.
...
Yes, Apple and Google (and Amazon, and Microsoft) control their respective app stores. But the difference from Dixon’s AOL analogy is that they don’t control the internet — and they don’t control each other. Apple doesn’t want cool new apps launching Android-only, and it surely bothers Google that so many cool new apps launch iOS-first. Apple’s stance on Bitcoin hasn’t exactly kept Bitcoin from growing explosively. App Stores are walled gardens, but the apps themselves are just clients to the open web/internet.
...
The rise of mobile apps hasn’t taken anything away from the wide open world of web browsers and cross-platform HTML/CSS/JavaScript — other than supremacy. I think that bothers some, who saw the HTML/CSS/JavaScript browser-centric web’s decade-ago supremacy as the end point, the ultimate triumph of a truly open platform, rather than what it really was: just another milestone along the way of an industry that is always in flux, ever ebbing and flowing.

What we’ve gained, though, is a wide range of interaction capabilities that never could have existed in a web browser-centric world. That to me is cause for celebration.

This difference is important to spell out because although HTTP and the Internet are key parts of the world wide web, they aren’t the web. One of the key things we lose with apps is public addressability (i.e. URIs for the technically inclined). What does this mean in practice

• Visiting a website is as simple as being told “go to http://bing.com” from any browser on any platform using any device. Getting an app requires the app developer to have created an app for your platform which may not have occurred due to technical limitations, policy limitations of the platform owner or simply the cost of supporting multiple platforms being higher than they want to bear.

• Content from apps is often invisible to search engines like Google and Bing since their information is not part of the web.

• Publishing a website simply requires getting a web host or even just hosting your own server. Publishing an app means submitting your product to some corporation then restricting your content and functionality to their rules & regulations before being made available to end users.

The key loss being that we are regressing from a globally accessible information network which reaches everyone on earth and where no publisher needs permission to reach billions of people to lots of corporate controlled fiefdoms and walled gardens.

I don’t disagree with Gruber’s notion that mobile apps have introduced new models of interaction that would not have existed in a web-browser centric world. However that doesn’t mean we aren’t losing something along the way.

Now Playing: The Heavy – Short Change Hero

I hate it when something I look at on Amazon turns up in a Facebook ad. It doesn't help me, it creeps me.

— Steven Levy (@StevenLevy) February 17, 2014

As part of my day job at Microsoft, I've begun to learn more about how advertising across the internet works on a technical level and it is quite interesting to learn how an image of a some head phones I looked at an e-commerce site ended up staring back at me from an ad on Facebook later that day.

The fundamental technology that makes this possible is Facebook Exchange (FBX). The infographic below provides an overview of how it enables ads from ecommerce sites to show up on Facebook and I’ll follow that up with a slightly more technical explanation.

Source: Business Insider

Facebook Exchange is a Real-Time Bidding platform which enables Facebook to sell ad slots on their page to the highest bidding advertisers in fractions of a second. Typically advertisers and publishers who own the pages where ads show up end up working together through an intermediary called a Demand Side Platform (DSP). A DSP such as AdRoll provides one of their retail partners such as American Apparel or Skull Candy with code to put tracking pixels on their site which allows the user to be identified and context such as what pages they’ve visited to be recorded. The retail partner then goes into AdRoll’s interface and decides how much they are willing to pay to show ads on various networks such as Facebook (via FBX) if a user who has visited one of their pages is shown an ad.

AdRoll then provides data to Facebook that allows the user to be uniquely identified within Facebook’s network. Later when that same user goes to Facebook, Facebook puts out a request on its Ad Exchange saying “Here’s a user who you might be interested in, how much are you willing to pay to show them an ad?”, AdRoll then cross-references that user’s opaque identifier with the behavioral data they have (i.e. what pages they were looking at on an advertiser’s site) and if there is a match they make a bid which will also include their ad for the page that piqued their interest. If the retailer wins the auction, then their ad is chosen and either rendered in the news feed or on the right hand side on Facebook’s desktop website. Each of these pieces needs to happen in fractions of a second but is still slow enough that rendering ads tends to noticeably be the slowest part of rendering the webpage.

In fact, there was a grand example of retargeting in action while I was researching this blog post. When I started writing this blog post I checked out AdRoll’s web page on how to use their service to retarget ads on Facebook. A few minutes later, this showed up in my news feed.

You can tell if an ad is retargeted on Facebook by hovering with your mouse cursor on the top right of the ad (on the desktop website) and then selecting the options. If the “About This Ad” link takes you somewhere outside Facebook then it is a retargeted ad.

Some ad providers like Quantcast provide an option to opt-out of retargeting for their service while others like AdRoll link to the Network Advertising Initiative (NAI) opt –out tool which provides an option to opt-out of retargeting for a variety of ad providers. Note that this doesn’t prevent you from getting ads and is instead just a signal to advertisers that you’d rather not have your ads personalized.

Now Playing: Ice Cube – Hood Mentality

Earlier this week my Twitter feed was flooded with reactions to the announcement of Amazon Prime Air. A vision which is compelling and sounds like something from a science fiction novel.

Flying robots delivering your impulse buys ordered from you smartphone within 30 minutes? Sign me up. 

Amazon's Robotic Vision

A few have been skeptical of Amazon Prime Air and some such as Konstantin Kakaes at Wired have described the announcement as an hour long infomercial hosted by Charlie Rose that is full of hot air. There are definitely a lot of things that need to get better before drone-based package delivery is a reality. On the technology end there are challenges such as improving navigation software and the getting more efficiency out of battery power. On the regulatory end the rules and regulations needed to ensure the safety of the populace in the midst of these flying drones still needs to be figured out.

Unlike a number of the skeptics, I'm quite confident that a number of the technological and regulatory hurdles will be surmounted in the next 5 years. On the other hand, I also believe Amazon oversold the technology. The logistics of actually delivering a package to a customer has been glossed over quite a bit. If you look at the video, this is the optimal situation for delivering a package.

Now think about all the places in urban environments that don't meet this criteria; apartment complexes, condos, office building, etc. Then think about all the places where packages are best left on the doorstep close to the house which would be infeasible for a drone to land close to. In fact it's likely the 30 minute delivery claim is meant to address some logistics challenges by assuming you'll be there to pick up anything the drone drops off on your lawn or driveway before Johnny Sticky Fingers does.

My suspicion is that the true impactful usage of Amazon Prime Air will be 30 minute delivery drone delivery to Amazon Locker locations.

Google's Robotic Vision

This morning my Twitter feed is abuzz with the news that Google's Andy Rubin, creator of Android and longtime robotics enthusiast, has acquired seven robotics companies that are creating technologies to build a mobile dexterous robot. This effort will be part of Google X which is also the home of Google Glass & self-driving cars. Andy Rubin describes the effort as follows

“I have a history of making my hobbies into a career,” Mr. Rubin said in a telephone interview. “This is the world’s greatest job. Being an engineer and a tinkerer, you start thinking about what you would want to build for yourself.”

He used the example of a windshield wiper that has enough “intelligence” to operate when it rains, without human intervention, as a model for the kind of systems he is trying to create. That is consistent with a vision put forward by the Google co-founder Larry Page, who has argued that technology should be deployed wherever possible to free humans from drudgery and repetitive tasks.

To spell it out, Google’s efforts in automation under project X have been about augmenting the human experience in ways that eliminate repetitive tasks which humans are poor at such as driving where human error is a significant cause of the over 30,000 deaths a year in the US due to automobile accidents. Even this somewhat frivolous example of ringing the doorbell at Andy Rubin’s home taken from a 2007 profile of the Android founder is consistent with that vision

If the scanner recognizes you, the door unlocks automatically. (The system makes it easier to deal with former girlfriends, Mr. Rubin likes to joke. No messy scenes retrieving keys — it’s just a simple database update.)

Those forced to use the doorbell are greeted with another technological marvel: a robotic arm inside the glass foyer grips a mallet and then strikes a large gong. Although Mr. Rubin won’t reveal its cost, it may be one of the world’s most expensive doorbells.

At the end of the day, I’m more inspired by a company looking at automating away all of the tediousness of every day life with Star Trek style technology from automated doorways and self-driving cars to fully autonomous robots than a vision of making impulse buying at Walmart 2.0 Amazon more convenient. I for one welcome our new robotic overlords.

Now Playing: Adele – Rumor Has It

I read an interesting blog post by Steven Levy titled Google Glass Team: ‘Wearable Computing Will Be the Norm’ with an interview with the Project Glass team which contains the following excerpt

Wired: Do you think this kind of technology will eventually be as common as smart phones are now?

Lee: Yes. It’s my expectation that in three to five years it will actually look unusual and awkward when we view someone holding an object in their hand and looking down at it. Wearable computing will become the norm.

The above reminds me of the Bill Gates quote, “there's a tendency to overestimate how much things will change in two years and underestimate how much change will occur over 10 years”. Coincidentally the past week has been full of retrospectives on the eve of the fifth anniversary of the iPhone. The iPhone has been a great example of how we can both overestimate and underestimate the impact of a technology. When the iPhone was announced as the convergence of an iPod, a phone and an internet mobile communicator the most forward thinking assumptions were that the majority of the Apple faithful who bought iPods would be people who bought iPhones and this would head off the demise of the iPod/MP3 player market category.

Five years later, the iPhone has effectively reshaped the computing industry. The majority of tech news today can be connected back to companies still dealing with the fallout of the creation of the iPhone and it’s progeny, the iPad. Entire categories of products across multiple industries have been made obsolete (or at least redundant) from yellow pages and paper maps to PDAs, point-and-shoot cameras and netbooks. This is in addition to the sociological changes that have been wrought (e.g. some children now think a magazine is a broken iPad). The most shocking change as a techie has been watching usage and growth of the World Wide Web being replaced by usage of mobile apps. No one really anticipated or predicted this five years ago.

Wearable computing will follow a similar path. It is unlikely that within a year or two of products like Project Glass coming to market that people will stop using smartphones especially since there are many uses for the ubiquitous smartphone that Project Glass hasn’t tried to address (e.g. playing Angry Birds or Fruit Ninja at the doctor’s office while waiting for your appointment). However it is quite clear that in our lifetime there will be the ability to put together scenarios that would have seemed far fetched for science fiction just a few years ago. It will one day be possible to look up the Facebook profile or future equivalent of anyone you meet at a bar, business meeting or on the street without the person being none the wiser simply by looking at them. Most of the technology to do this already exists, it just isn’t in a portable form factor. That is just one scenario that not only will be possible but will be commonplace with products like Project Glass in the future.

Focusing on Project Glass making smartphones obsolete is like focusing on the fact that the iPhone made iPod competitors like the Creative Zen Vision obsolete. Even if it did, that was not the interesting impact. As a software professional, it is interesting to ask yourself whether your product or business will be one of those obsoleted by this technology or empowered by it. Using analogies from the iPhone era, will you be RIM or will you be Instagram?

PS: I have to wonder what Apple thinks of all of this. When I look at the image below, I see a clunky and obtrusive piece of headgear that I can imagine makes Jonathan Ive roll his eyes and think he could do much better. Given Apple’s mantra is “If you don’t cannibalize yourself, someone else will” I expect this space to be very interesting over the next ten years.

Now Playing: Wale - Bag of Money (featuring Rick Ross, Meek Mill and T-Pain)

I stumbled on an interesting blog post today titled Why Files Exist which contains the following excerpt

Whenever there is a conversation about the future of computing, the discussion inevitably turns to the notion of a “File.” After all, most tablets and phones don’t show the user anything that resembles a file, only Apps that contain their own content, tucked away inside their own opaque storage structure.

This is wrong. Files are abstraction layers around content that are necessary for interoperability. Without the notion of a File or other similar shared content abstraction, the ability to use different applications with the same information grinds to a halt, which hampers innovation and user experience.

Given that one of the hats I wear in my day job is responsibility for the SkyDrive API, questions like whether the future of computing should include an end user facing notion of files and how interoperability across apps should work are often at the top of my mind. I originally wasn’t going to write about this blog post until I saw the discussion on Hacker News which was a bit disappointing since people either decided to get very pedantic on the specifics of how a computer file is represented in the operating system or argued that inter-app sharing between apps via intents (on Android) or contracts (in Windows 8/Windows RT) makes the notions of files obsolete.

The app-centric view of data (as espoused by iOS) is that apps own any content created within the app and there is no mechanism outside the app’s user experience to interact with or manage this data. This also means there is no global namespace where other apps or the end user can interact with this data also known as a file system. There are benefits to this approach such as greatly simplifying the concepts the user has to deal with and preventing both the user or other apps from mucking with the app’s experience. There are also costs to this approach as well.

The biggest cost is as highlighted in the Why Files Exist post is that interoperability is compromised. The reason is that it is a well known truism that data outlives applications. My contact list, my music library and the code for my side projects across the years are all examples of data which has outlived the original applications I used to create and manage them. The majority of this content is in the cloud today primarily because I want universal access to my data from any device and any app. A world where moving from Emacs to Visual Studio or WinAmp to iTunes means losing my files created in those applications would be an unfortunate place to live in the long term.

App-to-app sharing as is done with Android intents or contracts in Windows 8 is a powerful way to create loosely coupled integration between apps. However there is a big difference between one off sharing of data (e.g. share this link from my browser app to my social networking app) to actual migration or reuse of data (e.g. import my favorites and passwords from one browser app to another). Without a shared global namespace that all apps can access (i.e. a file system) you cannot easily do the latter.

The Why Files Exist ends with

Now, I agree with Steve Jobs saying in 2005 that a full blow filesystem with folders and all the rest might not be necessary, but in every OS there needs to be at least some user-facing notion of a file, some system-wide agreed upon way to package content and send it between applications. Otherwise we’ll just end up with a few monolithic applications that do everything poorly.

Here I actually slightly disagree with characterizing the problem as needing a way to package content and send it between applications. Often my data is actually conceptually independent of an application and it is more like I want to give access to my data to apps not that I want to package up some of my data from one app to another. For example, I wouldn’t characterize playing my MP3s originally ripped in Winamp or bought from Amazon MP3 in iTunes as packaging content between those apps and iTunes. Rather there is a global concept known as my music library which multiple apps can add to or play from.

So back to the question that is the title of this blog post; have files outlived their usefulness? Only if you think reusing data across multiple applications has.

Now Playing: Meek Mill - Pandemomiun (featuring Wale and Rick Ross

This past weekend, I bought an Apple iPhone as a replacement for my AT&T Tilt which was slowly succumbing to hardware glitches. As a big fan of Windows Mobile devices and their integration with Microsoft Exchange I was wary of adopting an iPhone. I was concerned about it not having all of the features I'd gotten used to but on the other hand I was looking forward to replacing my iPod+AT&T Tilt combination with a single device.

Here are my positive and negative impressions of the device after five days

The Good

There are lots of things I like about the iPhone experience. Below are my favorite aspects of the experience thus far

• Visual Polish: The visual polish of the 1st and 3rd party applications on the iPhone is amazing. There are so many nice touches in the phone from the coverflow browsing experience when playing music to the transparent pop over windows when you receive an SMS text. The few 3rd party applications I've used have also seemed similarly polished although I've only downloaded less than ten apps. It's like using a futuristic device that you see in movies not an actual phone from real life. 

• Browsing and Purchasing Applications: As someone whose used Handango and ActiveSync to install applications on my Windows Mobile device, I have to say that experience pales in comparison to being able to browse, search, purchase and install apps directly from my phone. Having the app store integrated into iTunes actually seems superfluous.

• It's also an iPod: I used to think my iPod classic was a fantastic music playing device until I got my iPhone. Now it seems rather primitive and ugly. I shocked at how much better the music playing experience is on my iPhone and have tossed my iPod classic into our junk drawer.  Now my pockets are lighter and I got an iPod upgrade. Nice. 

• The Web browser: The browser supports multi-touch for zoom and it does AJAX. Hell, yeah.

• Autocomplete when sending emails: When sending a mail, it uses autocomplete to fill out the TO/CC/BCC line by looking in your contact list and the email addresses of people in your inbox. This is a very nice touch.

The Bad

There are also a number of features I miss from owning a Windows Mobile device which I hope are addressed in the future or I might eventually find myself switching back

• Email Search: Windows Mobile devices can search emails in your Exchange inbox by sending a query to the server. Using this functionality you aren't limited to searching the emails on your device but can search at least month of emails and get the results sent down to your phone. The iPhone has no email search functionality.

• No integration with the Global Address List: On my AT&T Tilt I often needed to send emails to co-workers whose email addresses weren't in my contact list. All I needed to do was type out their names and then I could pull up their information (email address, phone number and office location) down to my phone to either add them as a contact or insert their email address into an email. I've felt rather handicapped without this functionality.  The autocomplete feature which uses all the email addresses from your local inbox has been a slight mitigation.

• No Flash in the browser: After getting used having a Flash-capable browser on my AT&T Tilt via Skyfire it is rather irritating that I've now lost that functionality by switching to an iPhone. You don't really notice how much Flash video content there is on the Web until you start missing it. My last post was of a Flash video which is a broken link when I browse my blog from my iPhone. Lame.

• Managing meetings is awful: As a program manager at Microsoft I schedule a lot of meetings. Every once in a while I may be running late for a meeting and have to either send a mail out to the attendees telling them I'll be late or cancel the meeting. Neither of these options is available on the iPhone.

• Reply flags not set in Exchange: With a Windows Mobile phone, when you respond to a mail via the phone it is properly marked as a mail you've replied to when you view it in Outlook on the desktop. The iPhone developers remembered to track which emails you've responded to on the device but failed to propagate that information back to Exchange. For a while, I thought I was going senile because I remembered responding to mails but they weren't marked as being replied to in my inbox. After I found my replies in my Sent mail folder, I realized what was wrong. 

• Tasks: Although I've never tried Getting Things Done, I am a big fan of Outlook Tasks and often add new tasks or mark them as complete from my phone. The iPhone does not synchronize your Outlook tasks from Exchange which is a glaring oversight. For now, I've gotten around this by spending $10 on KeyTasks for the iPhone which is somewhat hacky but works. 

The Ugly

So far there have been two aspects of the user experience that have been facepalm worthy

• Ringtones: On my AT&T Tilt it was pretty straightforward to make any MP3 snippet eligible to be used as a ring tone simply by dropping it in the right folder. The iPhone requires that I pay $0.99 for a song I already own to use it as a ring tone. Seriously?

• Using your iPhone on multiple computers: I typically purchase music and burn CDs on my home computer while using my iPod as a music library at work. This functionality is disabled out of the box on the iPhone. You can only sync your iPhone to one computer which includes only being able to play music off of your iPhone on a single computer. This is pretty ridiculous given multicomputer households and people who use their iPhones at home and at work. Thankfully, the Internet is full of workarounds to this foolishness on Apple's part. 

Despite what looks like a long list of complaints this is probably the best mobile phone I've ever owned. It just isn't perfect.

Now Playing: Jay-Z - Can't Knock the Hustle

From Palm Pre and Palm WebOS in-depth look we learn

The star of the show was the new Palm WebOS. It's not just a snazzy new touch interface. It's a useful system with some thoughtful ideas that we've been looking for. First of all, the Palm WebOS takes live, while-you-type searching to a new level. On a Windows Mobile phone, typing from the home screen initiates a search of the address book. On the Palm WebOS, typing starts a search of the entire phone, from contacts through applications and more. If the phone can't find what you need, it offers to search Google, Maps and Wikipedia. It's an example of Palm's goal to create a unified, seamless interface.

Other examples of this unified philosophy can be found in the calendar, contacts and e-mail features. The Palm Pre will gather all of your information from your Exchange account, your Gmail account and your Facebook account and display them in a single, unduplicated format. The contact listing for our friend Dave might draw his phone number from our Exchange account, his e-mail address from Gmail and Facebook, and instant messenger from Gtalk. All of these are combined in a single entry, with a status indicator to show if Dave is available for IM chats.

This is the holy grail of contact management experiences on a mobile phone. Today I use Exchange as the master for my contact records and then use tools like OutSync to merge in contact data for my Outlook contacts who are also on Facebook before pushing it all down to my Windows Mobile phone (the AT&T Tilt). Unfortunately this is a manual process and I have to be careful of creating duplicates when importing contacts from different places.

If the Palm Pre can do this automatically in a "live" anmd always connected way without creating duplicate or useless contacts (e.g. Facebook friends with no phone or IM info shouldn't take up space in my phone contact list) then I might have to take this phone for a test drive.

Anyone at CES get a chance to play with the device up close?

Now Playing: Hootie & The Blowfish - Only Wanna Be With You

This morning, Jeff Atwood wrote a blog post about software piracy entitled My Software Is Being Pirated where he talks about how companies can deal with the fact that the piracy rate among their users could be as high as 90%. He writes

Short of ..

1. selling custom hardware that is required to run your software, like the Playstation 3 or Wii
2. writing a completely server-side application like World of Warcraft or Mint

.. you have no recourse. Software piracy is a fact of life, and there's very little you can do about it. The more DRM and anti-piracy devices you pile on, the more likely you are to harm and alienate your paying customers. Use a common third party protection system and it'll probably be cracked along with all the other customers of that system. Nobody wants to leave the front door to their house open, of course, but you should err on the side of simple protection whenever possible. Bear in mind that a certain percentage of the audience simply can't be reached; they'll never pay for your software at any price . Don't penalize the honest people to punish the incorrigible. As my friend Nathan Bowers so aptly noted :

Every time DRM prevents legitimate playback, a pirate gets his wings.

In fact, the most effective anti-piracy software development strategy is the simplest one of all:

1. Have a great freaking product.
2. Charge a fair price for it.

(Or, more radically, choose an open source business model where piracy is no longer a problem but a benefit -- the world's most efficient and viral software distribution network. But that's a topic for a very different blog post.)

It is interesting to note that Jeff's recommendation for an effective anti-piracy solution is actually contradicted by the example game from his post; World of Goo. The game is an excellent product and is available for ~$15 yet it is still seeing a 90% piracy rate. In fact, the most effective anti-piracy strategy is simply to route around the problem as Jeff originally stated. Specifically

• target custom hardware platforms such as the iPhone or XBox 360 which don't have a piracy problem
• build Web-based software

However if you do decide to go down the shrinkwrapped software route, I'd suggest casting a critical eye on any claims that highlight the benefits of the "Open Source business model" to shrinkwrapped software developers. Open Source software companies have been around for over a decade (e.g. RedHat was founded in 1995) and we now have experience as an industry with regards to what works and what doesn't work as a business model for Open Source software.

There are basically three business models for companies that make money from Open Source software, they are

1. Selling support, consulting and related services for the "free" software (aka the professional open source business model ) – RedHat
2. Dual license the code and then sell traditional software licenses to enterprise customers who are scared of the GPL – MySQL AB
3. Build a proprietary Web application powered by Open Source software – Google

As you scan this list, it should be clear that none of these business models actually involves making money directly from selling only the software. This is problematic for developers of shrinkwrapped, consumer software such as games because none of the aforementioned business models actually works well for them.

For developers of shrinkwrapped software, Open Source only turns piracy from a problem into a benefit if you're willing to forego building consumer software and you have software that is either too complicated to use without handholding OR you can scare a large percentage of your customers into buying traditional software licenses by using the GPL instead of the BSDL.

Either way, the developers of World of Goo are still screwed.

Now Playing: The Notorious B.I.G. - Mo Money Mo Problems (feat. Mase & Puff Daddy)

A couple of months ago, Russell Beattie wrote a post about the end of his startup entitled The end of Mowser which is excerpted below

The argument up to now has been simply that there are roughly 3 billion phones out there, and that when these phones get on the Internet, their vast numbers will outweigh PCs and tilt the market towards mobile as the primary web device. The problem is that these billions of users *haven't* gotten on the Internet, and they won't until the experience is better and access to the web is barrier-free - and that means better devices and "full browsers". Let's face it, you really aren't going to spend any real time or effort browsing the web on your mobile phone unless you're using Opera Mini, or have a smart phone with a decent browser - as any other option is a waste of time, effort and money. Users recognize this, and have made it very clear they won't be using the "Mobile Web" as a substitute for better browsers, rather they'll just stay away completely.
…
In fact, if you look at the number of page views of even the most popular mobile-only websites out there, they don't compare to the traffic of popular blogs, let alone major portals or social networks.

Let me say that again clearly, the mobile traffic just isn't there. It's not there now, and it won't be.

What's going to drive that traffic eventually? Better devices and full-browsers. M-Metrics recently spelled it out very clearly - in the US 85% of iPhone owners browsed the web vs. 58% of smartphone users, and only 13% of the overall mobile market. Those numbers *may* be higher in other parts of the world, but it's pretty clear where the trend line is now. (What a difference a year makes.) It would be easy to say that the iPhone "disrupted" the mobile web market, but in fact I think all it did is point out that there never was one to begin with.

I filed away Russell's post as interesting at the time but hadn't really experienced it first hand until recently. I recently switched to using Skyfire as my primary browser on my mobile phone and it has made a world of difference in how a use my phone. No longer am I restricted to crippled versions of popular sites nor do I have to lose features when I visit the regular versions of the page. I can view the real version of my news feed on Facebook. Vote up links in reddit or Digg. And reading blogs is no longer an exercise in frustration due to CSS issues or problems rendering widgets. Unsurprisingly my usage of the Web on my phone has pretty much doubled.

This definitely brings to the forefront how ridiculous of an idea it was to think that we need a "mobile Web" complete with its own top level domain (.mobi). Which makes more sense, that every Web site in the world should create duplicate versions of their pages for mobile phones and regular browsers or that software + hardware would eventually evolve to the point where I can run a full fledged browser on the device in my pocket? Thanks to the iPhone, it is now clear to everyone that this idea of a second class Web for mobile phones was a stopgap solution at best whose time is now past. 

One other thing I find interesting is treating the iPhone as a separate category from "smartphones" in the highlighted quote. This is similar to a statement made by Walt Mossberg when he reviewed Google's Android. That article began as follows

In the exciting new category of modern hand-held computers — devices that fit in your pocket but are used more like a laptop than a traditional phone — there has so far been only one serious option. But that will all change on Oct. 22, when T-Mobile and Google bring out the G1, the first hand-held computer that’s in the same class as Apple’s iPhone.

The key feature that the iPhone and Android have in common that separates them from regular "smartphones" is that they both include a full featured browser based on Webkit. The other features like downloadable 3rd party applications, wi-fi support, rich video support, GPS, and so on have been available on phones running Windows Mobile for years. This shows how important having a full Web experience was for mobile phones and just how irrelevant the notion of a "mobile Web" has truly become.

Now Playing: Kilo Ali - Lost Y'all Mind

Given the current spirit of frugality that fills the air due to the credit crises I'm reconsidering whether to replace my AT&T Tilt (aka HTC Kaiser) with an iPhone 3G. After test driving a couple of iPhones I've realized that the really compelling reason for me to switch is to get a fully-featured Web browser instead of my current situation of having to choose between text-based "mobile" versions of popular sites or mangled Web pages.

I was discussing this with a coworker and he suggested that I try out alternative browsers for the Windows Mobile before getting an iPhone. I'd previously tried DeepFish from the Microsoft Live Labs team but found it too buggy to be usable. I looked for it again recently but it seems it has been cancelled. This led me to try out SkyFire which claims to give a complete PC-style Web browsing experience [including Flash Video, AJAX, Silverlight and Quicktime video] on a mobile phone.

After using SkyFire for a couple of days, I have to admit that it is a much improved Web browsing experience compared to what shipped by default on my phone. At first I marveled at how a small startup could build such a sophisticated browser in what seems like a relatively short time until I learned about the clever hack which is at the center of the application. None of the actual rendering and processing of content is done on your phone. Instead, there is an instance of a Web browser (supposedly Firefox) running on the SkyFire servers which acts as a proxy for your phone and then sends you a compressed image of the fully rendered results. There is still some clever hackery involved especially with regards to converting a streaming Flash video into a series of animated image and accompanying sound then sending it down to your phone in real-time. However it is nowhere near as complex as shipping complete Javascript, Flash, Quicktime and Silverlight implementations on mobile phone browser. 

The one problem with SkyFire's approach is that all of your requests go through their servers. This means your passwords, emails, bank account records or whatever other Web sites you visit with your mobile browser will flow through SkyFire's servers. This may be a deal breaker for some while for others it will mean being careful about what sites they visit using the browser. 

If this sounds interesting, check out the video demo below

Now Playing: Michael Jackson - PYT (Pretty Young Thing)

With the releases of betas of Google Chrome and Internet Explorer 8 as well as the recent release of Firefox 3, the pundits are all in a tizzy about the the new browser wars. I don't know if it is a war or not but I do like the fact that in the past few months we've seen clear proof that the end user experience when browsing the Web is going to get an upgrade for the majority of Web users.

Whenever there is such active competition between vendors, customers are typically the ones that benefit and the "new browser wars" are no different. Below are some of the features and trends in the new generation of browsers that has me excited about the future of the Web browsing user experience

One Process Per Tab

As seen in: IE 8 beta, Chrome

With this feature browsers are more resilient to crashes since each tab has its own process so a bug which would cause the entire browser to crash in an old school browser only causes the user to lose the tab in next generation browser. This feature is called Loosely Coupled IE (LCIE) in Internet Explorer 8 and described in the documentation of the Chrome Process Manager in the Google Chrome Comic Book.

This feature will be especially welcome for users of add-ons and browser toolbars since the IE team has found that up to 70% of browser crashes are caused by extensions and now these crashes will no longer take down the entire browser.

Smarter Address Bars

As seen in: IE 8 beta, Chrome, Firefox 3

Autocomplete in browser address bars has been improved. Instead of trying to match a user entered string as the start of a URL (e.g. "cn" autocompletes to http://cnn.com) newer browsers match any occurrence of the string in previously seen URLs and page titles (e.g. "cn" matches http://cnn.com, http://google.cn and a blog post on Wordpress with the title "I've stopped watching CNN").  Like Mark Pilgrim, I was originally suspicious of this feature but now cannot live without it.

This feature is called AwesomeBar in Firefox 3, OmniBox in Google Chrome and Smart Address Bar in IE 8.

Restoring Previous Browsing Sessions

As seen in: Firefox 3, Chrome, IE 8 beta

I love being able to close my browser and restart my operating system safe in the knowledge that whenever I launch the browser it is restored to exactly where I left off. Both Firefox and Chrome provide an option to make this behavior the default but the closest I've seen to getting a similar experience in the betas of IE 8 requires a click from the "about:Tabs" page. However given "about:Tabs" is my start page it gives maximum flexibility since I don't have to be slowed down by the opening up the four or five previously open browser tabs every time I launch my browser.

Search Suggestions

As seen in: IE 8 beta, Chrome, Firefox 3

In the old days, the only way to get search suggestions when typing a search query in your browser's search box was if you had a vendor specific search toolbar installed (e.g. Google Suggest for Firefox). It is becoming more commonplace for this to be native functionality of the Web browser. Google Chrome supports this if the default search provider is Google.  IE 8 beta goes one better by making this feature a platform that any search engine can plug into and currently provides search suggestions for the following search providers; Wikipedia, Amazon, Google, Live Search and Yahoo! as at this writing. 

Updated: Firefox has also supported search suggestions using a provider model since Firefox 2 via OpenSearch and ships with suggestions enabled for Google and Yahoo! by default.

Offline Support

As seen in: Chrome, IE 8 beta, Firefox 3

The WHAT WG created specifications which describes secure mechanisms for Web applications to store large amounts of user data on a local system using APIs provided by modern Web browsers. Applications can store megabytes of data on the user's local machine and have it accessible via the DOM. This feature was originally described in the Web Applications 1.0 specification and is typically called DOM Storage. You can read more about it in the Mozilla documentation for DOM Storage and the IE 8 beta documentation for DOM Storage. The related APIs are currently being defined as part of HTML 5.

Chrome supports this functionality by bundling Google Gears which is a Google defined set of APIs for providing offline storage. 

The most interesting thing about this list is that if you follow the pronouncements from various pundits on sites like Techmeme, you'd think all of these features were originated by Google and appeared for the first time in Chrome.

Update: An amusing take on the pundit hype about Google Chrome from Ted Dziuba in The Register article Chrome-fed Googasm bares tech pundit futility

Now Playing: Metallica - Cyanide

Last week my blog was offline for a day or so because I was the victim of a flood of SQL injection attacks that are still hitting my Web site at the rate of multiple requests a second. I eventually managed to counter the attacks by installing URLScan 3.0 and configuring it to reject HTTP requests that resemble SQL injection attacks. I found out about URLScan in two ways; from a blog post Phil Haack wrote about Dealing with Denial of Service Attacks where it seems he's been caught up in the same wave of attacks that brought down my blog and via an IM from Scott Hanselman who saw my tweet on Twitter about being hacked and pointed me to his blog post on the topic entitled Hacked! And I didn't like it - URLScan is Step Zero.

This reminded me that I similarly found another useful utility, WinDirStat, via a blog post as well. In fact when i think about it, a lot of the software I end up trying out is found via direct or indirect recommendations from people I know. Typically through blog posts, tweets or some other communication via a social networking or social media service. This phenomenon can be clearly observed in closed application ecosystems like the Facebook platform, where statistics have shown that the majority of users install new applications after viewing them on the profiles of their friends.

One of the things I find most interesting about the Facebook platform and now the Apple App Store is that they are revolutionizing how we think about software distribution. Today, finding interesting new desktop/server/Web apps either happens serendipitously via word of mouth or [rarely] is the result of advertising or PR. However finding interesting new applications if you are a user of Facebook or the Apple iPhone isn't a matter of serendipity. There are well understood ways of finding interesting applications that harnesses social and network effects from user ratings to simply finding out what applications your friends are using.

As a user, I sometimes wish I had an equivalent experience as a user of desktop applications and their extensions. I've often thought it would be cool to be able to browse the software likes and dislikes of people such as Omar Shahine, Scott Hanselman and Mike Torres to see what their favorite Windows utilities and mobile applications were. As a developer of a feed reader, although it is plain to see that Windows has a lot of reach since practically everyone runs it sometimes I'm envious of the built in viral distribution features that come with the Facebook platform or the unified software distribution experience that is the iPhone App Store. Sure beats hosting your app on SourceForge and hoping that your users are blogging about the app to spread it via word of mouth or paying for prominence on sites like Download.com.

A lot of the pieces are already there. Microsoft has a Windows Marketplace but for the life of me I'd have never found out about it if I didn't work at Microsoft and someone I know switched teams to start working there. There are also services provided by 3rd parties like Download.com, the Firefox Add-Ons page and Tucows. It would be interesting to see what could be stitched together if you throw in a social graph via something like Facebook Connect, an always-on well integrated desktop experience similar to the Apple App Store and one of the aforementioned sites. I suspect the results would be quite beneficial to app developers and users of Windows applications.

What do you think?

Now Playing: Metallica - The Day That Never Comes

Disclaimer: This post does not reflect the opinions, thoughts, strategies or future intentions of my employer. These are solely my personal opinions. If you are seeking official position statements from Microsoft, please go here.

Earlier this week, David Recordon announced the creation of the Open Web Foundation at OSCON 2008. His presentation is embedded below

From the organization's Web site you get the following outline of it's mission

The Open Web Foundation is an attempt to create a home for community-driven specifications. Following the open source model similar to the Apache Software Foundation, the foundation is aimed at building a lightweight framework to help communities deal with the legal requirements necessary to create successful and widely adopted specification.

The foundation is trying to break the trend of creating separate foundations for each specification, coming out of the realization that we could come together and generalize our efforts. The details regarding membership, governance, sponsorship, and intellectual property rights will be posted for public review and feedback in the following weeks.

Before you point out that this seems to create yet another "standards" organization for Web technology, there are already canned answers to this question. Google evangelist Dion Almaer provides justification for why existing Web standards organizations do not meet their needs in his post entitled The Open Web Foundation; Apache for the other stuff where he writes 

Let’s take an example. Imagine that you came up with a great idea, something like OAuth. That great idea gains some traction and more people want to get involved. What do you do? People ask about IP policy, and governance, and suddenly you see yourself on the path of creating a new MyApiFoundation.

Wait a minute! There are plenty of standards groups and other organizations out there, surely you don’t have to create MyApiFoundation?

Well, there is the W3C and OASIS, which are pay to play orgs. They have their place, but MyApi may not fit in there. The WHATWG has come up with fantastic work, but the punting on IP is an issue too.

At face value, it's hard to argue with this logic. The W3C charges fees using a weird progressive taxation model where a company pays anything from a few hundred to several thousand dollars depending on how the W3C assesses their net worth. OASIS similarly charges from $1,000 to $50,000 depending on how much influence the member company wants to have in the organization. After that it seems there are a bunch of one off organizations like the Open ID foundation and the WHATWG that are dedicated to a specific technology. 

Or so the spin from the Open Web Foundation would have you believe.

In truth there is already an organization dedicated to producing "Open" Web technologies that has a well thought out policy on membership, governance, sponsorship and intellectual property rights that isn't pay to play. This is not a new organization, it actually happens to be older than David Recordon who unveiled the Open Web Foundation.

The name of this organization is the Internet Engineering Task Force (IETF). If you are reading this blog post then you are using technologies for the "Open Web" created by the IETF. You may be reading my post in a Web browser in which case the content was transferred to you over HTTP (RFC 2616) and if you're reading it in an RSS reader then I should add that you're also directly consuming my Atom feed (RFC 4287). Some of you are reading this post because someone sent you an email which is another example of an IETF protocol at work, SMTP (RFC 2821).

The IETF policy on membership doesn't get more straightforward; join a mailing list. I am listed as a member of the Atom working group in RFC 4287 because I was a participant in the atom-syntax mailing list. The organization has a well thought out and detailed policy on intellectual property rights as it relates the IETF specifications which is detailed in RFC 3979: Intellectual Property Rights in IETF Technology and slightly updated in RFC 4879: Clarification of the Third Party Disclosure Procedure in RFC 3979.

I can understand that a bunch of kids fresh out of college are ignorant of the IETF and believe they have to reinvent the wheel to Save the Open Web but I am surprised that Google which has had several of it's employees participate in the IETF processes which created RFC 4287, RFC 4959, RFC 5023 and RFC 5034 would join in this behavior. Why would Google decide to sponsor a separate standards organization that competes with the IETF that has less inclusive processes than the IETF, no clear idea of how corporate sponsorship will work and a yet to be determined IPR policy?

That's just fucking weird.

Now Playing: Boyz N Da Hood - Dem Boys (remix) (feat T.I. & The Game)

For the past few years, the technology press has been eulogizing desktop and server-based software while proclaiming that the era of Software as a Service (SaaS) is now upon us. According to the lessons of the Innovator's Dilemma the cheaper and more flexible SaaS solutions will eventually replace traditional installed software and the current crop of software vendors will turn out to be dinosaurs in a world that belongs to the warm blooded mammals who have conquered cloud based services.

So it seems the answer is obvious, software vendors should rush to provide Web-based services and extricate themselves from their "legacy" shrinkwrapped software business before it is too late. What could possibly go wrong with this plan? 

Sarah Lacy wrote an informative article for Business Week about the problems facing software vendors who have rushed into the world of SaaS. The Business Week article is entitled On-Demand Computing: A Brutal Slog and contains the following excerpt

On-demand represented a welcome break from the traditional way of doing things in the 1990s, when swaggering, elephant hunter-style salesmen would drive up in their gleaming BMWs to close massive orders in the waning days of the quarter. It was a time when representatives of Oracle ( ORCL ), Siebel, Sybase ( SY ), PeopleSoft, BEA Systems, or SAP ( SAP ) would extol the latest enterprise software revolution, be it for management of inventory, supply chain, customer relationships, or some other area of business. Then there were the billions of dollars spent on consultants to make it all work together—you couldn't just rip everything out and start over if it didn't. There was too much invested already, and chances are the alternatives weren't much better.

Funny thing about the Web, though. It's just as good at displacing revenue as it is in generating sources of it. Just ask the music industry or, ahem, print media. Think Robin Hood, taking riches from the elite and distributing them to everyone else, including the customers who get to keep more of their money and the upstarts that can more easily build competing alternatives.

But are these upstarts viable? On-demand software has turned out to be a brutal slog. Software sold "as a service" over the Web doesn't sell itself, even when it's cheaper and actually works. Each sale closed by these new Web-based software companies has a much smaller price tag. And vendors are continually tweaking their software, fixing bugs, and pushing out incremental improvements. Great news for the user, but the software makers miss out on the once-lucrative massive upgrade every few years and seemingly endless maintenance fees for supporting old versions of the software.
…
Nowhere was this more clear than on Oracle's most recent earnings call (BusinessWeek.com, 6/26/08). Why isn't Oracle a bigger player in on-demand software? It doesn't want to be, Ellison told the analysts and investors. "We've been in this business 10 years, and we've only now turned a profit," he said. "The last thing we want to do is have a very large business that's not profitable and drags our margins down." No, Ellison would rather enjoy the bounty of an acquisition spree that handed Oracle a bevy of software companies, hordes of customers, and associated maintenance fees that trickle straight to the bottom line.
…
SAP isn't having much more luck with Business by Design, its foray into the on-demand world, I'm told. SAP said for years it would never get into the on-demand game. Then when it sensed a potential threat from NetSuite, SAP decided to embrace on-demand. Results have been less than stellar so far. "SAP thought customers would go to a Web site, configure it themselves, and found the first hundred or so implementations required a lot of time and a lot of tremendous costs," Richardson says. "Small businesses are calling for support, calling SAP because they don't have IT departments. SAP is spending a lot of resources to configure and troubleshoot the problem."

In some ways, SaaS vendors have been misled by the consumer Web and have failed to realize that they still need to spend money on sales and support when servicing business customers. Just because Google doesn't advertise it's search features and Yahoo! Mail doesn't seem to have a huge support staff that hand holds customers as it uses their product doesn't mean that SaaS vendors can expect to cut their sales and support calls. The dynamics of running a free, advertising based service aimed at end users is completely different from running a service where you expect to charge business tens of thousands to hundreds of thousands to use your product.

In traditional business software development you have three major cycles with their own attendant costs; you have to write the software, you have to market the software and then you have to support the software. Becoming a SaaS vendor does not eliminate any of these costs. Instead it adds new costs and complexities such as managing data centers and worrying about hackers. In addition, thanks to free advertising based consumer services and the fact that companies like Google that have subsidized their SaaS offerings using their monopoly profits in other areas, business customers expect Web-based software to be cheaper than its desktop or server-based alternatives. Talk about being stuck between a rock and a hard place as a vendor.

Finally, software vendors that have existing ecosystems of partners that benefit from supporting and enhancing their shrinkwrapped products also have to worry about where these partners fit in a SaaS world. For an example of the kinds of problems these vendors now face, below is an excerpt from a rant by Vladimer Mazek, a system administrator at ExchangeDefender, entitled Houston… we have a problem which he wrote after attending one of Microsoft's partner conferences

Lack of Partner Direction: By far the biggest disappointment of the show. All of Microsoft’s executives failed to clearly communicate the partnership benefits. That is why partners pack the keynotes, to find a way to partner up with Microsoft. If you want to gloat about how fabulous you are and talk about exciting commission schedules as a brand recommender and a sales agent you might want to go work for Mary Kay. This is the biggest quagmire for Microsoft – it’s competitors are more agile because they do not have to work with partners to go to market. Infrastructure solutions are easy enough to offer and both Google and Apple and Amazon are beating Microsoft to the market, with far simpler and less convoluted solutions. How can Microsoft compete with its partners in a solution ecosystem that doesn’t require partners to begin with?

This is another example of the kind of problems established software vendors will have to solve as they try to ride the Software as a Service wave instead of being flattened by it.  Truly successful SaaS vendors will eventually have to deliver platforms that can sustain a healthy partner ecosystems to succeed in the long term. We have seen this in the consumer space with the Facebook platform and in the enterprise space with SalesForce.com's AppExchange. Here is one area where the upstarts that don't have a preexisting shrinkwrap software businesses can turn a disadvantage (lack of an established partner ecosystem) into an advantage since it is easier to start from scratch than to retool.

The bottom line is that creating a Web-based version of a popular desktop or server-based product is just part of the battle if you plan to play in the enterprise space. You will have to deal with the sales and support that go with selling to businesses as well as all the other headaches of shipping "cloud based services" which don't exist in the shrinkwrap software world. After you get that figured out, you will want to consider how you can leverage various ISVs and startups to enhance the stickiness of your service and turn it into a platform before one of your competitor's does. 

I suspect we still have a few years before any of the above happens. In the meantime we will see lots more software companies complaining about the paradox of embracing the Web when it clearly cannibalizes their other revenue streams and is less lucrative than what they've been accustomed to seeing. Interesting times indeed.

Now Playing: Flobots - Handlebars

Last week TechCrunch UK wrote about a search startup that utilizes AI/Semantic Web techniques named True Knowledge. The post entitled VCs price True Knowledge at £20m pre-money. Is this the UK’s Powerset?  stated

The chatter I’m hearing is that True Knowledge is being talked about in hushed tones, as if it might be the Powerset of the UK. To put that in context, Google has tried to buy the Silicon Valley search startup several times, and they have only launched a showcase product, not even a real one. However, although True Knowledge and Powerset are similar, they are different in significant ways, more of which later.
...
Currently in private beta, True Knowledge says their product is capable of intelligently answering - in plain English - questions posed on any topic. Ask it if Ben Affleck is married and it will come back with "Yes" rather than lots of web pages which may or may not have the answer (don’t ask me!).
...
Here’s why the difference matters. True Knowledge can infer answers that the system hasn’t seen. Inferences are created by combining different bits of data together. So for instance, without knowing the answer it can work out how tall the Eiffel Tower is by inferring that it is shorter that the Empire State Building but higher than St Pauls Cathedral.
...
AI software developer and entrepreneur William Tunstall-Pedoe is the founder of True Knowledge. He previously developed a technology that can solve a commercially published crossword clues but also explain how the clues work in plain English. See the connection?

The scenarios described in the TechCrunch write up should sound familiar to anyone who has spent any time around fans of the Semantic Web. Creating intelligent agents that can interrogate structured data on the Web and infer new knowledge has turned out to  be easier said than done because for the most part content on the Web isn't organized according to the structure of the data. This is primarily due to the fact that HTML is a presentational language. Of course, even if information on the Web was structured data (i.e. idiomatic XML formats) we still need to build machinary to translate between all of these XML formats.

Finally, in the few areas on the Web where structured data in XML formats is commonplace such as Atom/RSS feeds for blog content, not a lot has been done with this data to fulfill the promise of the Semantic Web.

So if the Semantic Web is such an infeasible utopia, why are more and more search startups using that as the angle from which they will attack Google's dominance of Web search? The answer can be found in Bill Slawski's post from a year ago entitled Finding Customers Through Anti-Commercial Queries where he wrote

Most Queries are Noncommercial

The first step might be to recognize that most queries conducted by people at search engines aren't aimed at buying something. A paper from the WWW 2007 held this spring in Banff, Alberta, Canada, Determining the User Intent of Web Search Engine Queries , provided a breakdown of the types of queries that they were able to classify.

Their research uncovered the following numbers: "80% of Web queries are informational in nature, with about 10% each being navigational and transactional." The research points to the vast majority of searches being conducted for information gathering purposes. One of the indications of "information" queries that they looked for were searches which include terms such as: “ways to,” “how to,” “what is.”

Although the bulk of the revenue search engines make is from people performing commercial queries such as searching for "incredible hulk merchandise", "car insurance quotes" or "ipod prices", this is actually a tiny proportion of the kinds of queries people want answered by search engines. The majority of searches are about the five Ws (and one H) namely "who", "what", "where", "when", "why" and "how". Such queries don't really need a list of Web pages as results, they simply require an answer. The search engine that can figure out how to always answer user queries directly on the page without making the user click on half a dozen pages to figure out the answer will definitely have moved the needle when it comes to the Web search user experience.

This explains why scenarios that one usually associates with AI and Semantic Web evangelists are now being touted by the new generation of "Google-killers". The question is whether knowledge inference techniques will prove to be more effective than traditional search engine techniques when it comes to providing the best search results especially since a lot of the traditional search engines are learning new tricks.

Now Playing: Bob Marley - Waiting In Vain

Disclaimer: This post does not reflect the opinions, thoughts, strategies or future intentions of my employer. These are solely my personal opinions. If you are seeking official position statements from Microsoft, please go here.

About a month ago Joel Spolsky wrote a rant about Microsoft's Live Mesh project which contained some interesting criticism about the project and showed that Joel has a personal beef with the Googles & Microsofts of the world for making it hard for him to hire talented people to work for his company. Unsurprisingly, lots of responses focused on the latter since it was an interesting lapse in judgement for Joel inject his personal frustrations into what was meant to be a technical critique of a software project. However there were some ideas worthy of discussion in Joel's rant that I've been pondering over the past month. The relevant parts of Joel's article are excerpted below

It was seven years ago today when everybody was getting excited about Microsoft's bombastic announcement of Hailstorm, promising that "Hailstorm makes the technology in your life work together on your behalf and under your control."

What was it, really? The idea that the future operating system was on the net, on Microsoft's cloud, and you would log onto everything with Windows Passport and all your stuff would be up there. It turns out: nobody needed this place for all their stuff. And nobody trusted Microsoft with all their stuff. And Hailstorm went away.
...
What's Microsoft Live Mesh?

Hmm, let's see.

"Imagine all your devices—PCs, and soon Macs and mobile phones—working together to give you anywhere access to the information you care about."

Wait a minute. Something smells fishy here. Isn't that exactly what Hailstorm was supposed to be? I smell an architecture astronaut.

And what is this Windows Live Mesh?

It's a way to synchronize files.

Jeez, we've had that forever. When did the first sync web sites start coming out? 1999? There were a million versions. xdrive, mydrive, idrive, youdrive, wealldrive for ice cream. Nobody cared then and nobody cares now, because synchronizing files is just not a killer application. I'm sorry. It seems like it should be. But it's not.

But Windows Live Mesh is not just a way to synchronize files. That's just the sample app. It's a whole goddamned architecture, with an API and developer tools and in insane diagram showing all the nifty layers of acronyms, and it seems like the chief astronauts at Microsoft literally expect this to be their gigantic platform in the sky which will take over when Windows becomes irrelevant on the desktop. And synchronizing files is supposed to be, like, the equivalent of Microsoft Write on Windows 1.0.

As I read the above rant I wondered what world Joel has been living in the past half decade. Hailstorm has actually proven to have been a very visionary and accurate picture of how the world ended up. A lot of the information that used to sit in my desktop in 2001 is now in the cloud. My address book is on Facebook, my photos are on Windows Live Spaces and Flickr, my email is in Hotmail and Yahoo! Mail, while a lot of my documents are now on SkyDrive and Google Docs. Almost all of these services provide XML-based APIs for accessing my data and quite frankly I find it hard to distinguish the ideas behind a unified set of user-centric cloud APIs that was .NET My Services from Google GData. A consistent set of APIs for accessing a user's contact lists, calendar, documents, inbox and profile all stored on the servers of a single company? Sounds like we're in a time warp doesn't it? Even more interesting is that outlandish sounding scenarios at the time such as customers using a delegated authentication model to grant applications and Web sites temporary or permanent access to their data stored in the cloud are now commonplace. Today we have OAuth, Yahoo! BBAuth, Google AuthSub, Windows Live DelAuth and even just the plain old please give us your email password.

In hindsight the major problem with Hailstorm seems to have been that it was a few years ahead of its time and people didn't trust Microsoft. Funny enough, a lot of the key people who were at Microsoft during that era like Vic Gundotra and Mark Lucovsky are now at Google, a company and a brand which the Internet community trusts a lot more than Microsoft, working on Web API strategy. 

All of this is a long winded way of saying I think Joel's comparison of Live Mesh with Hailstorm is quite apt but just not the way Joel meant it. I believe that like Hailstorm, Live Mesh is a visionary project that in many ways tries to tackle problems that people will have or don't actually realize they have. And just like with Hailstorm where things get muddy is separating the vision from the first implementation or "platform experience" of that vision.

I completely agree with Joel that synchronizing files is not a killer application. It just isn't sexy and never will be. The notion of having a ring or mesh of devices where all my files synchronize across each device in my home or office is cool to contemplate from a technical perspective. However it's not something I find exciting or feel that I need even though I'm a Microsoft geek with a Windows Mobile phone, an XBox 360, two Vista laptops and a Windows server in my home. It seems I'm not the only one that feels that way according to a post by a member of the Live Mesh team entitled Behind Live Mesh: What is MOE? which states

Software + Services

When you were first introduced to Live Mesh, you probably played with the Live Desktop .  It’s pretty snazzy.  Maybe you even uploaded a few files too.  Hey, it’s a cool service!  You can store stuff in a cloud somewhere and access it anywhere using a webpage.  Great!

As I look at the statistics on the service though, I notice that a significant portion of our users have stopped here.  This pains me, as there’s a whole lot more you can do with Live Mesh.  Didn’t you hear all the hoopla about Software + Services ?  Ever wonder, “Where’s the software?”

You might have noticed that on the device ring there’s a big orange button with a white ‘+’ sign.  The magic happens when you click that big orange button and opt to “add a device” to your mesh. 

So what excites me as a user and a developer about Live Mesh? I believe seamless synchronization of data as a platform feature is really interesting. Today I use OutSync to add people's Facebook photos to their contact information on my Windows Mobile phone. I've written my own RSS reader which synchronizes the state of my RSS subscriptions with Google Reader. Doug Purdy wrote FFSync so he can share his photos, music taste and other data on his Mac with his friends on FriendFeed. It may soon be possible to synchronize my social graph across multiple sites via Google Friend Connect. Google is working on using Google Gears to give me offline access to my documents in Google Docs by synchronizing the state between my desktop and their cloud. Earlier this week Apple announced mobile.me which enables users to synchronize their contacts, emails, calendar and photos across the Web and all their devices.

Everywhere I look data synchronization is becoming more and more important and also more commonplace. I expect this trend to continue over time given the inevitable march of the Web. Being able to synchronize my data and my actions from my desktop to the Web or across Web sites I frequent is extremely enabling. Thus having a consistent  set of standards-based protocols for enabling these scenarios as well as libraries for the key platforms that make this approachable to developers will be very beneficial to users and to the growth of the Web. 

At the rate things are going, I personally believe that this vision of the Web will come to pass with or without Microsoft in the same way that Hailstorm's vision of the Web actually came to pass even though Microsoft canned the project. Whether Microsoft is an observer or a participant in this new world order depends on whether Live Mesh as a product and as a vision fully embraces the Web and collaboration with Web companies (as Google has ably done with GData/OpenSocial/FriendConnect/Gears/etc) or not. Only time will tell what happens in the end.

Now Playing: Usher - In This Club (remix) (feat. Beyonce & Lil Wayne)

I've been having problems with hard drive space for years. For some reason, I couldn't get over the feeling that I had less available space on my hard drive than I could account for. I'd run programs like FolderSizes and after doing some back of the envelope calculations it would seem like I should have gigabytes more free space than what was actually listed as available according to my operating system.

Recently I stumbled on a blog post by Darien Nagle which claimed to answer the question Where's my hard disk space gone? with the recommendation that his readers should try WinDirStat. Seeing nothing to lose I gave it a shot and I definitely came away satisfied. After a quick install, it didn't take long for the application to track down where all those gigabytes of storage I couldn't account for had gone. It seems there was a hidden folder named C:\RECYCLER that was taking up 4 GigaBytes of space.

I thought that was kind of weird so I looked up the folder name and found Microsoft KB 229041 - Files Are Not Deleted From Recycler Folder which listed the following symptoms

SYMPTOMS
When you empty the Recycle Bin in Windows, the files may not be deleted from your hard disk.

NOTE: You cannot view these files using Windows Explorer, My Computer, or the Recycle Bin.

I didn't even have to go through the complicated procedure in the KB article to delete the files, I just deleted them directly from the WinDirStat interface.

My only theory as to how this happened is that some data got orphaned when I upgraded my desktop from Windows XP to Windows 2003 since the user accounts that created them were lost. I guess simply deleting the files from Windows Explorer as I did a few years ago wasn't enough.

Good thing I finally found a solution. I definitely recommend WinDirStat, the visualizations aren't half bad either.

Now Playing: Eminem - Never Enough (feat. 50 Cent & Nate Dogg)