Joel Spolsky has an seminal article entitled Don't Let Architecture Astronauts Scare You where he wrote

A recent example illustrates this. Your typical architecture astronaut will take a fact like "Napster is a peer-to-peer service for downloading music" and ignore everything but the architecture, thinking it's interesting because it's peer to peer, completely missing the point that it's interesting because you can type the name of a song and listen to it right away.

All they'll talk about is peer-to-peer this, that, and the other thing. Suddenly you have peer-to-peer conferences, peer-to-peer venture capital funds, and even peer-to-peer backlash with the imbecile business journalists dripping with glee as they copy each other's stories: "Peer To Peer: Dead!"

 The Architecture Astronauts will say things like: "Can you imagine a program like Napster where you can download anything, not just songs?" Then they'll build applications like Groove that they think are more general than Napster, but which seem to have neglected that wee little feature that lets you type the name of a song and then listen to it -- the feature we wanted in the first place. Talk about missing the point. If Napster wasn't peer-to-peer but it did let you type the name of a song and then listen to it, it would have been just as popular.

This article is relevant because I recently wrote a series of posts explaining why Web developers have begun to favor JSON over XML in Web Services. My motivation for writing this article were conversations I'd had with former co-workers who seemed intent on "abstracting" the discussion and comparing whether JSON was a better data format than XML in all the cases that XML is used today instead of understanding the context in which JSON has become popular.

In the past two weeks, I've seen three different posts from various XML heavy hitters committing this very sin

  1. JSON and XML by Tim Bray - This kicked it off and starts off by firing some easily refutable allegations about the extensibility and unicode capabilities of JSON as a general data transfer format.
  2. Tim Bray on JSON and XML by Don Box - Refutes the allegations by Tim Bray above but still misses the point.
  3. All markup ends up looking like XML by David Megginson - argues that XML is just like JSON except with the former we use angle brackets and in the latter we use curly braces + square brackets. Thus they are "Turing" equivalent. Academically interesting but not terribly useful information if you are a Web developer trying to get things done.

This is my plea to you, if you are an XML guru and you aren't sure why JSON seems to have come out of nowhere to threaten your precious XML, go read JSON vs. XML: Browser Security Model and JSON vs. XML: Browser Programming Models then let's have the discussion.

If you're too busy to read them, here's the executive summary. JSON is a better fit for Web services that power Web mashups and AJAX widgets due to the fact it gets around the cross domain limitations put in place by browsers that hamper XMLHttpRequest and that it is essentially serialized Javascript objects which makes it fit better client side scripting which is primarily done in Javascript. That's it. XML will never fit the bill as well for these scenarios without changes to the existing browser ecosystem which I doubt are forthcoming anytime soon.

Update: See comments by David Megginson and Steve Marx below.


 

Categories: XML

It's a new year and time for another brand new Windows Live service to show up in beta. This time it's the Windows Live for TV Beta which is described as follows

What it is
Windows Live™ for TV Beta is a rich, graphically-driven interface designed for people who use Windows Live Spaces and Messenger and Live Call on large-screen monitors and TVs. We're still in the early stages of this beta, so many of the features might not work properly yet. That's why we really need your feedback! This beta is in limited release, so you must request access to the trial group. After you’re in the beta, come back to this page and let us know what you think.

You can also find out more about the product on the team's blog at http://wlfortv.spaces.live.com which includes the following screenshot

Hey, I think I can see me in that screen shot. :)


 

Categories: Windows Live

A perennial topic for debate on certain mailing lists at work is rich client (i.e. desktop) software versus Web-based software. For every person that sings the praises of Web-based program such as Windows Live Mail, there's someone to wag their finger who points out that "it doesn't work offline" and "not everyone has a broadband connection". A lot of these discussions have become permathreads on the some of the mailing lists I'm one and I can recite detailed arguments for both sides in my sleep.

However I think both sides miss the point and agree more than they disagree. The fact is that highly connected societies such as the North America and Western Europe computer usage overlaps almost completely with internet usage (see Nielsen statistics for U.S. homes and Top 25 most connected countries). This trend will only increase as internet penetration spreads across developing countries emerging markets. 

What is important to understand is that for a lot of computer users, their computer is an overpriced paperweight if it doesn't have an Internet connection. They can't read the news, can't talk to their friends via IM, can't download music to their iPods Zunes, can't people watch on Facebook or MySpace, can't share the pictures they just took with their digital cameras, can't catch up on the goings on at work via email, they can't look up driving directions, can't check the weather report, can't do research for any reports they have to write and the list goes on. Keeping in mind that connectivity is key is far more important than whether the user experience is provided via a desktop app written using Win32 or is a "Web 2.0" website powered by AJAX. Additionally, the value of approachability and ease of use over "features" and "richness" cannot be emphasized enough.

Taken from that perspective, a lot of things people currently consider "features" of desktop applications are actually bugs in todays Internet-connected world. For example, I have different files in the "My Documents" folders on the 3 or 4 PCs I use regularly. Copying files between PCs and keeping track of what version of what file is where is an annoyance. FolderShare to the rescue.

When I'm listening to my music on my computer I sometimes want to be able to find out what music my friends are listening to, recommend my music to friends or just find music similar to what I'm currently playing. Last.fm and iLike to the rescue.

The last time I was on vacation in Nigeria, I wanted to check up on what was going on at work but never had access to a computer with Outlook installed nor could I have actually set it up to talk to my corporate account even if I could. Outlook Web Access to the rescue.

Are these arguments for Web-based or desktop software? No. Instead they are meant to point out that improving the lives of computer users should mean finding better ways of harnessing their internet connections and their social connections to others. Sometimes this means desktop software,   sometimes it will mean Web-based software and sometimes it will be both.


 

Categories: Technology

Over the holidays I had a chance to talk to some of my old compadres from the XML team at Microsoft and we got to talking about the JSON as an alternative to XML. I concluded that there are a small number of key reasons that JSON is now more attractive than XML for kinds of data interchange that powers Web-based mashups and Web gadgets widgets. This is the second in a series of posts on what these key reasons are.

In my previous post, I mentioned that getting around limitations in cross domain requests imposed by modern browsers has been a key reason for the increased adoption of JSON. However this is only part of the story.

Early on in the adoption of AJAX techniques across various Windows Live services I noticed that even for building pages with no cross domain requirements, our Web developers favored JSON to XML. One response that kept coming up is the easier programming model when processing JSON responses on the client than with XML. I'll illustrate this difference in ease of use via a JScript code that shows how to process a sample document in both XML and JSON formats taken from the JSON website. Below is the code sample

var json_menu = '{"menu": {' + '\n' +
'"id": "file",' + '\n' +
'"value": "File",' + '\n' +
'"popup": {' + '\n' +
'"menuitem": [' + '\n' +
'{"value": "New", "onclick": "CreateNewDoc()"},' + '\n' +
'{"value": "Open", "onclick": "OpenDoc()"},' + '\n' +
'{"value": "Close", "onclick": "CloseDoc()"}' + '\n' +
']' + '\n' +
'}' + '\n' +
'}}';


var xml_menu = '<menu id="file" value="File">' + '\n' +
'<popup>' + '\n' +
'<menuitem value="New" onclick="CreateNewDoc()" />' + '\n' +
'<menuitem value="Open" onclick="OpenDoc()" />' + '\n' +
'<menuitem value="Close" onclick="CloseDoc()" />' + '\n' +
'</popup>' + '\n' +
'</menu>';

WhatHappensWhenYouClick_Xml(xml_menu);
WhatHappensWhenYouClick_Json(json_menu);

function WhatHappensWhenYouClick_Json(data){

  var j = eval("(" + data + ")");

  WScript.Echo("
When you click the " + j.menu.value + " menu, you get the following options");

  for(var i = 0; i < j.menu.popup.menuitem.length; i++){
   WScript.Echo((i + 1) + "." + j.menu.popup.menuitem[i].value
    + " aka " + j.menu.popup.menuitem[i].onclick);
  }

}

function WhatHappensWhenYouClick_Xml(data){

  var x = new ActiveXObject( "Microsoft.XMLDOM" );
  x.loadXML(data);

  WScript.Echo("When you click the " + x.documentElement.getAttribute("value")
                + " menu, you get the following options");

  var nodes = x.documentElement.selectNodes("//menuitem");

  for(var i = 0; i < nodes.length; i++){
   WScript.Echo((i + 1) + "." + nodes[i].getAttribute("value") + " aka " + nodes[i].getAttribute("onclick"));
  }
}

When comparing both sample functions, it seems clear that the XML version takes more code and requires a layer of mental indirection as the developer has to be knowledgeable about XML APIs and their idiosyncracies. We should dig a little deeper into this. 

A couple of people have already replied to my previous post to point out that any good Web application should process JSON responses to ensure they are not malicious. This means my usage of eval() in the code sample, should be replaced with JSON parser that only accepts 'safe' JSON responses. Given that that there are JSON parsers available that come in under 2KB that particular security issue is not a deal breaker.

On the XML front, there is no off-the-shelf manner to get a programming model as straightforward and as flexible as that obtained from parsing JSON directly into objects using eval(). One light on the horizon is that E4X becomes widely implemented in Web browsers . With E4X, the code for processing the XML version of the menu document above would be 

function WhatHappensWhenYouClick_E4x(data){

  var e = new XML(data);

  WScript.Echo("When you click the " + j.menu.value + " menu, you get the following options");

  foreach(var m in e.menu.popup.menuitem){
   WScript.Echo( m.@value + " aka " + m.@onclick);
  }

}

However as cool as the language seems to be it is unclear whether E4X will ever see mainstream adoption. There is an initial implementation of E4X in the engine that powers the Firefox browser which seems to be incomplete. On the other hand, there is no indication that either Opera or Internet Explorer will support E4X in the future.

Another option for getting the simpler object-centric programming models out of XML data could be to adopt a simple XML serialization format such as XML-RPC and providing off-the-shelf Javascript parsers for this data format. A trivial implementation could be for the parser to convert XML-RPC to JSON using XSLT then eval() the results. However it is unlikely that people would go through that trouble when they can just use JSON.

This may be another nail in the coffin of XML on the Web. 


 

Categories: Web Development | XML | XML Web Services

Over the holidays I had a chance to talk to some of my old compadres from the XML team at Microsoft and we got to talking about the JSON as an alternative to XML. I concluded that there are a small number of key reasons that JSON is now more attractive than XML for kinds of data interchange that powers Web-based mashups and Web gadgets widgets. This is the first in a series of posts on what these key reasons are.

The first "problem" that chosing JSON over XML as the output format for a Web service solves is that it works around security features built into modern browsers that prevent web pages from initiating certain classes of communication with web servers on domains other than the one hosting the page. This "problem" is accurately described in the XML.com article Fixing AJAX: XMLHttpRequest Considered Harmful which is excerpted below

But the kind of AJAX examples that you don't see very often (are there any?) are ones that access third-party web services, such as those from Amazon, Yahoo, Google, and eBay. That's because all the newest web browsers impose a significant security restriction on the use of XMLHttpRequest. That restriction is that you aren't allowed to make XMLHttpRequests to any server except the server where your web page came from. So, if your AJAX application is in the page http://www.yourserver.com/junk.html, then any XMLHttpRequest that comes from that page can only make a request to a web service using the domain www.yourserver.com. Too bad -- your application is on www.yourserver.com, but their web service is on webservices.amazon.com (for Amazon). The XMLHttpRequest will either fail or pop up warnings, depending on the browser you're using.

On Microsoft's IE 5 and 6, such requests are possible provided your browser security settings are low enough (though most users will still see a security warning that they have to accept before the request will proceed). On Firefox, Netscape, Safari, and the latest versions of Opera, the requests are denied. On Firefox, Netscape, and other Mozilla browsers, you can get your XMLHttpRequest to work by digitally signing your script, but the digital signature isn't compatible with IE, Safari, or other web browsers.

This restriction is a significant annoyance for Web developers because it eliminates a number of compelling end user applications due to the limitations it imposes on developers. However, there are a number of common workarounds which are also listed in the article

Solutions Worthy of Paranoia

There is hope, or rather, there are gruesome hacks, that can bring the splendor of seamless cross-browser XMLHttpRequests to your developer palette. The three methods currently in vogue are:

  1. Application proxies. Write an application in your favorite programming language that sits on your server, responds to XMLHttpRequests from users, makes the web service call, and sends the data back to users.
  2. Apache proxy. Adjust your Apache web server configuration so that XMLHttpRequests can be invisibly re-routed from your server to the target web service domain.
  3. Script tag hack with application proxy (doesn't use XMLHttpRequest at all). Use the HTML script tag to make a request to an application proxy (see #1 above) that returns your data wrapped in JavaScript. This approach is also known as On-Demand JavaScript.

Although the first two approaches work, there are a number of problems with them. The first is that it adds a requirement that the owner of the page also have Web master level access to a Web server and either tweak its configuration settings or be a savvy enough programmer to write an application to proxy requests between a user's browser and the third part web service. A second problem is that it significantly increases the cost and scalability impact of the page because the Web page author now has to create a connection to the third party Web service for each user viewing their page instead of the user's browser making the connection. This can lead to a bottleneck especially if the page becomes popular. A final problem is that if the third party service requires authentication [via cookies] then there is no way to pass this information through the Web page author's proxy due to browser security models.

The third approach avoids all of these problems without a significant cost to either the Web page author or the provider of the Web service. An example of how this approach is utilized in practice is described in Simon Willison's post JSON and Yahoo!’s JavaScript APIs where he writes

As of today, JSON is supported as an alternative output format for nearly all of Yahoo!’s Web Service APIs. This is a Really Big Deal, because it makes Yahoo!’s APIs available to JavaScript running anywhere on the web without any of the normal problems caused by XMLHttpRequest’s cross domain security policy.

Like JSON itself, the workaround is simple. You can append two arguments to a Yahoo! REST Web Service call:

&output=json&callback=myFunction

The page returned by the service will look like this:

myFunction({ JSON data here });

You just need to define myFunction in your code and it will be called when the script is loaded. To make cross-domain requests, just dynamically create your script tags using the DOM:

var script = document.createElement('script');
script.type = 'text/javascript';
script.src = '...' + '&output=json&callback=myFunction';
document.getElementsByTagName('head')[0].appendChild(script);

People who are security minded will likely be shocked that this technique involves Web pages executing arbitrary code they retrieve from a third party site since this seems like a security flaw waiting to happen especially if the 3rd party site becomes compromised. One might also wonder what's the point of browsers restricting cross-domain HTTP requests if pages can load and run arbitrary Javascript code [not just XML data] from any domain.

However despite these concerns, it gets the job done with minimal cost to all parties involved and more often than not that is all that matters.

Postscript: When reading articles like Tim Bray's JSON and XML which primarily compares both data formats based on their physical qualities, it is good to keep the above information in mind since it explains a key reason JSON is popular on the Web today which turns out to be independent of any physical qualities of the data format. 


 

Categories: Web Development | XML | XML Web Services

Over the holidays I had a chance to talk to some of my old compadres from the XML team at Microsoft and we got to talking about the JSON as an alternative to XML. I concluded that there are a small number of key reasons that JSON is now more attractive than XML for kinds of data interchange that powers Web-based mashups and Web gadgets widgets. Expect a series of posts on this later today. 

I wasn't sure I was going to write about this until I saw Mike Arrington's blog post about a GMail vulnerability which implied that this is another data point in the XML vs. JSON debate. On reading about the vulnerability on Slashdot I disagree. This seems like a novice cross site scripting vulnerability that is independent of JSON or XML, and is succintly described in the Slashdot comment by TubeSteak which states

Here's the super simple explanation

1. Gmail sets a cookie saying you're logged in
2. A [3rd party] javascript tells you to call Google's script
3. Google checks for the Gmail cookie
4. The cookie is valid
5. Google hands over the requested data to you

If [3rd party] wanted to
keep your contact list, the javascript would pass it to a form and your computer would happily upload the list to [3rd party]'s server.

Mitigations to this problem are also well known and are also summarized in another Slashdot comment by buro9 who writes

When you surface data via Xml web services, you can only call the web service on the domain that the JavaScript calling it originates from. So if you write your web services with AJAX in mind exclusively, then you have made the assumption that JavaScript is securing your data.

The problem is created at two points:
1) When you rely on cookies to perform the implicit authentication that reveals the data.
2) When you allow rendering of the data in JSON which bypasses JavaScript cross-domain security.

This can be solved by doing two things:
1) Make one of the parameters to a web service a security token that authenticates the request.
2) Make the security token time-sensitive (a canary) so that a compromised token does not work if sniffed and used later.
.

The surprising thing is that I'd assumed that the knowledge of using canary values was commonplace but it took a lot longer than I expected to find a good bite size description of them. And when I did, it came from co-worker Yaron Goland in a comment to Mark Nottingham's post on DOM vs. Web where Yaron wrote

There are a couple of ways to deal with this situation:

Canaries - These are values that are generated on the fly and sent down with pages that contain forms. In the previous scenario evil.com wouldn't know what canary site X was using at that instant for that user and so its form post wouldn't contain the right value and would therefore be rejected. The upside about canaries is that they work with any arbitrary form post. The downside is that they require some server side work to generate and monitor the canary values. Hotmail, I believe, uses canaries.

Cookies - A variant on canaries is to use cookies where the page copies a value from a cookie into the form before sending it up. Since the browser security model only allows pages from the same domain to see that domain's cookie you know the page had to be from your domain. But this only works if the cookie header value isn't easily guessable so in practice it's really just canaries.

XMLHTTP - Using XMLHTTP it's possible to add HTTP headers so just throw in a header of any sort. Since forms can't add headers you know the request came from XMLHTTP and because of XMLHTTP's very strict domain security model you know the page that sent the request had to come from your site.

I guess just because something is common knowledge among folks building Web apps and toolkits at Microsoft doesn't mean it is common knowledge on the Web. This is another one of those things that everyone building Web applications should know about to secure their applications but very few actually learn.


 

Categories: Web Development

December 31, 2006
@ 04:19 PM

Jeff Simmermon wrote a blog post entitled Drowning Kittens In A River Full Of Cash which he seems to have deleted but which is cached here. There was a sentence from that blog post which stayed with me and I have reproduced below

Forget about trying to write under that kind of dread. Writing under family-friendly corporate constraints is a necessary but curious clusterfuck in the best conditions. Sometimes it's like reaching deep within your soul and pulling out a basket of kittens, then quietly drowning it in a river. A man's gotta eat, though, and I never griped about the paycheck. That was my choice, and I made it every day.

This year was the first year I considered ending this blog because I'd finally gotten tired of the hassle of people complaining about what I wrote here. The final straw for me surprisingly hasn't been work related although there have been stretching points from disgruntled coworkers who lashed out because I use competing products to people complaining to my management chain and theirs hoping to get me reprimanded or even fired for not toeing the party line. I stand by everything I've written in this blog but I've now gotten enough heat and taken enough inter-personal communication training classes to realize that some opinions are more trouble than they are worth. So every once in a while, a quietly drown a kitten of a half written blog post because I can't be bothered with dealing with the feedback. However that wasn't the breaking point, since I've considered this experience part of "growing up".

What I didn't expect to have to deal with was people back home in Nigeria reading my blog. Or even worse, certain posts in from my blog being printed out and republished in Nigerian print magazines. That audience which now includes quite a few members of my family is one I hadn't anticipated and one whose feedback on misconstrued posts is one I take more to heart than the other kinds of feedback I'm used to getting about my blog. This has now introduced a new set of filters I have to apply to my blog posts.

I've now begun to question the purpose of continuing to write this blog and considered ending it and perhaps restarting an anonymous one on some generic blog hosting service like TypePad or Blogger. I'm not sure what I'm going to do next but thought it only fair to let the couple thousand folks who read this blog regularly to know why it stopped if it does stop.

Have a Happy New Year.
 

Categories: Personal

December 28, 2006
@ 03:52 PM

The Wired Vaporware Awards '06 was announced this week and item #8 is The IPod Killer. Specifically the article states

8. The "IPod Killer"

Every time we hear about a new portable audio device, it's touted as the magic bullet that will end the iPod's reign once and for all.

Oh, you mean that thing? The one with the ugly design and clunky user interface? Does it at least work with iTunes?

Microsoft's Zune was supposed to do the trick, but it's sitting on the shelves (even the SanDisk Sansa is selling better than Zune) while the kids are all enjoying their freshly unwrapped, shiny new iPods.

Note to everyone: It's a White Wire World. Get over it.

I've bought a lot of iPods in my day and I've been making a list of things I'd like to see out of Microsoft's Zune effort that would tip the scales and make us go from a multiple iPod household to a Zune household. Below is a list of the 5 things that I think it would take to convert myself, my girlfriend and her oldest daughter from iPod users to Zune users based on conversations we've had about switching.

  1. Improve the Pocketability Factor: If you check out pictures of an iPod and a Zune side by side you'll notice that the iPod is thinner and shorter. Since I carry an MP3 player around a few hours a day either around my arm when working out or when walking around at work, the less bulky it is the better. My girlfriend definitely doesn't want to walk around with the outline of a rectangular brick in her pants especially now that skinny jeans are in

  2. Do Something Cool with the Wifi: At Christmas dinner, I was talking to a family friend who just graduated from college and she asked me about the Zune. She perked up when she heard about the Wifi capabilities and music sharing until she found out the limitations in the music sharing capabilities (3 days or 3 plays) which she described as lame. I doubt that there is much that can be done to make the music sharing feature interesting to college kids since [as she told me] they just want free music. However there are a lot of cool things I can imagine doing with a portable music player that has Wifi. For example, what if I was listening to a song on the radio on my Zune then could purchase that song in a few clicks right from the device? What if I could receive internet radio with no ads on my device?  

  3. iPod Dock Connector Compatibility: A lot of people mistakenly think that the large investment that prevents people from switching from an iPod is the library of songs they've purchased from iTMS which won't be compatible with another DRM system. However in my experience I've noticed that the huge range of iPod dock connector accessories is becoming more of a factor. After all, auto manufacturers from Chrysler to Ford, GM and Mazda are all shipping iPod-ready cars. We spent a few hundred dollars converting my girlfriend's SUV to an iPod-ready one. Then there's all the travel speakers, including my Christmas present a FUNKit DJ iPod Speakers. Re-buying hundreds of dollars worth of accessories doesn't fill me with glee. However if we could go to an electronics store like RadioShack and get a an iPod<->Zune connector that would ease the transition a great deal.

  4. Smaller, Cheaper Versions: The iPod Nano has totally swept the young girl demographic. Whenever we end up at the mall and I'm dragged to Claire's I tend to see a lot of iPod Nano accessories but very few for the video iPod. We need a Zune that targets that demographic.

  5. Celebrity Product Placement: I almost feel embarrassed to admit this but the tipping point for getting my first iPod wasn't when all the gadget wunderkinds at work got one or when I almost fell of the treadmill because I was switching CDs on my portable CD player. No, the tipping point was when I saw 50 Cent using one in the P.I.M.P. video. If you want to impress me with the cool kids that are using the player, don't show me annoying looking hipsters I probably couldn't stand if I met in real life. Another good example of this kind of product placement is Mariah Carey using a RAZR in the Shake It Off video.

What would it take for you to switch to a Zune?
 

Categories: Music

December 27, 2006
@ 07:08 PM

Mark Cuban has a blog post entitled Ripping on Gootube... Again which introduced me to a Youtube trend I hadn't even noticed. He writes

Take a look at Decembers Top Viewed English Videos.
Most Viewed (This Month)
...
Add Video to QuickList
video
Added: 3 weeks ago
From: tylermcgregor
Views: 1408363
3523 ratings
Add Video to QuickList
video
Added: 1 week ago
From: VerizonWireless
Views: 1373397
5731 ratings
Add Video to QuickList
video
Added: 1 week ago
From: wylinoutbitch
Views: 1341496
3299 ratings
Add Video to QuickList
video
Added: 2 weeks ago
From: CBS
Views: 1277719
3895 ratings

Go through the list. Only the StarWars PSA, the Christmas Tree Jump and PowerTool Racing are really user generated content. 3 out of 20.

From there you have a contrived 12 days of christmas that is one of thousands of promos for Youtube users themselves trying to build a following. Is this social networking at its best ?

From there we have commercials or promos for movies, for tv shows, for blenders, knives, for music videos and for a phone company. Then we have the most popular of Youtube videos these days. The fake Porn thumbnail with headlines of. Britney, Paris, whoever, nude, in the shower, wherever, doing whatever. 5 of the top 20 are fake porn.

The fact that the professionally produced content is more popular on YouTube than amateur content isn't that surprising. By definition, professionals can put more time and effort into producing high quality content which is why their content is more popular. This is true in almost all areas of user generated content areas including blogging, see the Technorati Top 100 for proof.

What is surprising is how popular 'gotcha' content which pretends to be soft porn turns out to be a practical joke has become. The two fake porn videos linked above have been viewed over a million times. The interesting question is whether Google/Youtube will do anything to curtail this trend. This is likely a cause for user dissatisfaction on the site based on all the negative responses I saw in the comments to the videos, however there seem to be enough people who find it funny that this isn't a clear case of spam nor can it be voted down by the community since a lot of people may vote them up since they find them amusing. 

As Mark Cuban points out in his post, this is one of those perennial problems with social software. If trends or people that are harmful to the community show up there isn't a clear way to deal with them without it seeming like the heavy hand authority slapping down people trying to have fun and express themselves. On the other hand, I doubt Google spent $1.62 billion for Youtube just to watch it turn into a haven for fake porn and other forms of griefing.


 

Categories: Social Software

The bigger Google grows, the more it seems that like every large company it's products now have to pay a strategy tax which may end up shortchanging their users. If you are unfamiliar with the term, you should read Dave Winer's excellent essay Strategy Tax which is excerpted below

Ben explained that sometimes products developed inside a company such as Microsoft have to accept constraints that go against competitiveness, or might displease users, in order to further the cause of another product. I recognized the concept but had never heard the term.

An example. Consider a company that develops both a Web browser and a word processor. The product team for the browser might get lots of input from users saying "We'd like a better editor in the browser." It would be natural to give the users what they want. So they put the feature in their project plan, but when it comes up for review, the CEO shoots them down. "They should use the word processor," he says.

Another example. An electronics company makes a portable music player, and also owns a recording studio. The team working on the music player knows that there's a big market for players that work with a format that's popular with computer users, but lacks sophisticated copy protection. They project huge sales. But when the product comes up for review, the CEO shoots them down. "We'll lose sales of our music," he says.

Before I read this essay I thought this was a problem unique to Microsoft and also thought that I was being astute in observing the trend at the company when in truth the term was part of the cultural knowledge of the company while I was still in programming diapers. Over time, it has become clear to me that this is a problem that affects any business endeavor where different product units either rely on each other or indirectly compete with each other [assuming that the company isn't dysfunctional enough to have products that directly compete against each other]. Below are three examples of how the strategy tax is affecting Google, all of which are observations by other bloggers which I've noticed myself but refrained from mentioning since I work for a competitor and it would have come off as sour grapes.

Disincentive to Improve Search Due to Ad Business

In his post Good Luck Jimmy, Dave Winer writes

Google is repeating the pattern of the previous generation of search engines (Alta Vista, Infoseek) were doing when Google zigged to their zag, so successfully. Today, Google is fattening up and spreading out, going after Microsoft in productivity apps, chasing the TV networks with YouTube. Etc etc. Today search is only one of the things Google is doing, and it may not be the most important thing.

Today Google's profits come from ads, and that business gives them a reason to keep search weak. They want you to do a lot of searching to find what you're looking for -- and the stuff they find for you for free is competing with the stuff they make money on. So Google actually has a disincentive to make search better.

A few months ago, I used to get into regular debates with Matt Augustine who argued that the fact that companies like Google make so much money from search advertising seems like a bug in the system. Matt would argue that if search engines were really that good at finding what we want, we would never have to click on the results they had been paid to show us unless we were deceived into doing so.

This seems to put a 'glass ceiling' on how good the search engine can be because you don't want people to stop clicking on ads when you make billions of dollars a year from them doing so.

Promoting Google Services at the Expense of the Integrity of Search Results and it's Advertisers

Blake Ross has a blog post entitled Tip: Trust is hard to gain, easy to lose where he writes

But Google lost me today, and it didn’t take much:

Google is now displaying “tips” that point searchers to Google Calendar, Blogger and Picasa for any search phrase that includes “calendar” (e.g. Yahoo calendar), “blog” and “photo sharing,” respectively. This is clearly bad for competitors, and it’s also a bad sign for Google. But I generally support anything that benefits users, including monopolistic packaging. I believe, for instance, that shipping Internet Explorer with Windows was a good move. So why are tips bad for users?
...
The tips are different—and bad for users—because the services they recommend are not the best in their class. If Google wants to make it faster and easier for users to manage events, create a blog or share photos, it could do what it does when you search GOOG: link to the best services. To prevent Google from being the gatekeeper, the company could identify the services algorithmically. But if that sounds familiar, perhaps that’s because Google already works that way. After all, Google is predicated on the idea that the democratic structure of the Web will push the cream to the top. Search for “photo sharing” and you should already get the highest quality services. According to Google, Picasa is not one of them.
...
While advertisers compete to be first in a string of lookalike ads that are often shunted to the side, Google now determines the precise position and appearance of ads tips that are not subject to any of the same rules. Its ads get icons while others don’t, and if you think that’s small potatoes, you are not an advertiser: images boost clickthrough. Google can make a Picasa ad say “Easier to use than Kodak,” but Kodak cannot create an ad that reads “Easier to use than Picasa.” And the kicker: neither the highest quality ads nor the highest quality search results can replace these tips.

The "strategy tax" here is being paid by the search engine and advertising groups at Google. To carry along Google services that Blake points out are not best in class, Google is foregoing ad dollars from a number of lucrative keywords and causing distrust in the search engine by the very power users upon whose backs it rose to fame in the first place. Google used to brag about how unlike other search engines, they don't use misleading ads that people can confuse for search results. However I tend to agree with the last statement in Blake's post

Perhaps the most nefarious aspect of this feature is how it operates within our collective blind spots. Advertisers are happy that Google no longer invades the canonical Ad Results. Technology purists continue to see untainted Search Results. But does my mother make that distinction? How much does a result have to look like a Result to cross the line?
Indeed.

Artificially Promoting it's Products in Search Results

From a comment highlighted in the post Google's Silent Monopoly Redux (Google Responds - Issues Public Statement) which states

But type in "maps". Google is again first. Ahead of Mapquest. Ahead of Yahoo maps. Yahoo also has backlinks out the ying yang. So why is it third? And mapquest has been around forever.. I'm sure there are more links to mapquest.com than to the maps.google.com URL, simply because the millions of web pages that linked their directions to Mapquest from 1996 to 2004 didn't all rush out and change all their links to Google maps in February of 2005 (when it was released), even if Google's is a better product.

Next, try "mail". Despite the fact that Yahoo mail has been around forever, and has all sorts of links, and that Hotmail has also been around forever, Gmail still manages to come up first.

And the most interesting thing about this particular keyword? The word "mail" by itself doesn't even appear on the gmail.google.com page! The words gmail, webmail, and email appear. But not "mail". At least on the Yahoo page, the word "mail" does indeed appear. Yet Google still manages to rank ahead of Yahoo.

Finally, try "answers". Yes, answers.google.com comes up second, rather than first. But answers.yahoo.com comes in third! Is the Google Answers site really getting that many more links than Yahoo's? Especially in light of the fact that Google recently decided to kill it, because almost no one was using it, while Yahoo's usage (and therefore also linkage, no doubt) are skyrocketing?

This claim was actually the most interesting to me since Google is very adamant about the integrity of their search results and claims we don’t accept payment for inclusion in our index, nor do we manipulate search results by hand. I tried a number of these queries myself and was pretty shocked by the results especially when it came to "mail". Here are some screenshots that illustrate the point

1. Search results for "mail" on Google

2. Number of links to gmail.google.com (also the same as gmail.com) according to Google

3. Number of links to mail.yahoo.com according to Google

It's hard to imagine any objective metric that should make Gmail show up ahead of Yahoo! Mail in a search for the word "mail". Of course, this doesn't mean that Google is tampering with search results "by hand". Their algorithm can simply have allowances to rank sites in their domain or linked from their domain higher without having to actually sully their hands by tweaking individual results by hand. Still, if Google is how the world finds information and we are increasingly being pointed to information that financially benefits Google, doesn't that taint the much vaunted claim of the integrity of their search results even if it is being done in an automated manner?