Greg Linden has a blog post entitled Early Amazon: Shopping Cart Recommendations where he writes

I loved the idea of making recommendations based on the items in your Amazon shopping cart. Add a couple things, see what pops up. Add a couple more, see what changes. The idea of recommending items at checkout is nothing new. Grocery stories put candy and other impulse buys in the checkout lanes. Hardware stores put small tools and gadgets near the register. But here we had an opportunity to personalize impulse buys. It is as if the rack near the checkout lane peered into your grocery cart and magically rearranged the candy based on what you are buying.
...
I hacked up a prototype. On a test site, I modified the Amazon.com shopping cart page to recommend other items you might enjoy adding to your cart. Looked pretty good to me. I started showing it around. While the reaction was positive, there was some concern. In particular, a marketing senior vice-president was dead set against it. His main objection was that it might distract people away from checking out -- it is true that it is much easier and more common to see customers abandon their cart at the register in online retail -- and he rallied others to his cause.

At this point, I was told I was forbidden to work on this any further. I was told Amazon was not ready to launch this feature. It should have stopped there. Instead, I prepared the feature for an online test. I believed in shopping cart recommendations. I wanted to measure the sales impact.

I heard the SVP was angry when he discovered I was pushing out a test. But, even for top executives, it was hard to block a test. Measurement is good. The only good argument against testing would be that the negative impact might be so severe that Amazon couldn't afford it, a difficult claim to make. The test rolled out
...
The results were clear. Not only did it win, but the feature won by such a wide margin that not having it live was costing Amazon a noticeable chunk of change. With new urgency, shopping cart recommendations launched.

On the O'Reilly Radar site, Marc Hedlund points to this post as an example company leadership that encourages employees to not only come up with innovations but go head-to-head with management to get them out to customers.

It's interesting how different people look at the same story. When I originally read the story, what jumped out to me was that Amazon must have a great A/B testing framework which allows them to measure such tweaks to the user experience of their site so accurately. Coincidentally, I just had lunch with one of the folks at work who is building a generic A/B testing framework for Windows Live, MSN and third party developers; the Microsoft Experimentation Platform.

I wonder how many web companies have the infrastructure to test and measure the kind of change Greg prototyped at Amazon on their website today? Probably not a lot. We should fix that.


 

Wednesday, 24 May 2006 23:11:08 (GMT Daylight Time, UTC+01:00)
Hey, Dare,

Good point. I agree that A/B testing is extremely important for any organization that wants to have measurement trump other factors. You might also be interested in an earlier Radar post that talked about that more extensively:

http://radar.oreilly.com/archives/2006/02/web_development_20.html

Regards,
Marc
Comments are closed.