I attended the panel on Open Source Infrastructure hosted by Marc Canter, Tantek Çelik,
Brian Dear, Matt Mullenweg and Toni Schneider.

Marc Canter coined the term "Open Source infrastructure" while pitching OurMedia to big companies while seeking funding. He pointed out that in the Web 2.0 world we are in danger of swapping the lock-in of desktop platforms controlled by big companies like Apple and Microsoft for lock-in of Web platforms controlled by big companies like eBay and Amazon. The same way we have open source platforms to prevent desktop lock-in, we need open source Web infrastructure to prevent platform lock-in.

Brian Dear of EVDB is working on making event publishing on the Web easier. Eventful is a website that aggregates events. The website is built on the EVDB API which is itself built on an the EVDB index over the EVDB Database. This same API is available for third parties to build applications against. Brian divides events into high definition and low definition events. Low definition events are easy to create and have simple metadata usually just a title and start/end time for the event. However simple events are hard to discover due to the lack of structured metadata. On the other hand, high definition events have lots of fields (title, start/end time, description, price, recurrence, etc) which makes them harder to create but easier to discover by applications.  They have created SES (simple event sharing) which is a mechanism for web sites to ping an event server with changes similar to how weblogs currently ping places like Weblogs.com and blo.gs when they are updated.  

At this point Marc Canter interjects and asks where the events are located. Will they be able to suck up events from sites like Craig's List or will they have to be on Eventful? Brian Dear states that he prefers the model where aggregators of events such as Eventful points to existing sites such as Craig's List especially since they are not metadata rich (i.e. low definition events). Marc then points to someone in the audience who has a similar site and asks whether they have a ping server model and he mentions that they crawl the Web instead.  

This segued into Matt Mullenweg talking about ping servers. Matt talked about ping-o-matic which aggregates ping servers so blogs can just ping it and it pings the dozens of ping servers out there instead. However they have significant scaling issues with some days where they get up to 4,000,000 pings a day. Unsurprisingly, a lot of the pings turn out to be spam. Matt has asked help from various sources and has gotten servers from Technorati. . Marc asks whether pings can grow beyond blogs to events, group creation and other types of pings. Although Matt seemed hesitant, Brian points out that they have extended the ping format already from what sites like Weblogs.com uses to accomodate their needs for events.

Yahoo! just bought upcoming which is an event site and Tony Schneider [who used to be at Oddpost] was representing Yahoo! on the panel. Tony believes that big companies shouldn't own the core services on the Web which is one of the motivations for Yahoo! opening up their APIs. The biggest develop communities around Yahoo!'s offerings come from the entities they have purchased such as Flickr & Konfabulator. Turning their services into developer platforms is now big at Yahoo!. Another thing that is really big at Yahoo! is using RSS. Yahoo! doesn't care that it isn't an official standard produced by a standard's body. It gets the job done. They are also acquiring core online services like blo.gs and keeping them open to benefit the developer world.  Marc asks when they decide when working with others (e.g. MediaRSS) versus buying companies (blo.gs and upcoming). Todd replies that with formats they first look at whether anything exists before creating something new (e.g. GeoRSS used in Yahoo! Maps API vs. MediaRSS which they created).

Tantek talked about Web 2.0 themes ('you control your own data', 'mix and match data', 'open, interoperable and web friendly data formats & protocols'). Marc points out that the Web is not the end all and be all so 'Web friendly' is cool but not overriding. Tantek also discussed Talked about microformat design principles and the various themes within the microformats community (open source tools, open communications and focus on concrete scenarios). He then briefly talked about two microformats. The first was hReview which is a format for reviews which got input from folks at Yahoo!, SixApart and MSN among others.  there are a lot of websites for reviews (Amazon, Yahoo! Local) but no real standard. The second was hReview which is now being used by http://www.avon.com to mark up the contact information for over 40,000 Avon representatives.  Tantek also shows that you can mark up the Web 2.0 speakers list and then he wrote a bookmarklet that can suck up all the speakers into his address book.

During the Q & A session I asked three questions 

1. Isn't the focus on centralized services for pinging worrying for services with lots of users because it is quite possible for us to overwhelm the services with our user base? Matt responded that this is why he was seeking donations from large companies. 
2. Currently microformats seem to be going fine since Tantek is the main guy creating them but what happens when others start creating them and unlike XML which has namespaces there is no way to disambiguate them? Tantek responded that there were already many people creating microformats. He also stated that the issue of duplicate or redundant microformats would be dealt with via the community process.
3. Isn't the big problem with the lack of adoption of standards for creating events, the lack of a killer app for consuming events? Tantek responded that the killer apps may be here already by showing how he wrote an app to consume hCalendar events and place them in his iCal calendar. Brian mentioned that Eventful uses hCalendar and hCard. 

In general, I'd have preferred if the panel was more of a discussion as opposed to an hour or more of sales pitches from the panelists with 10 minutes of Q & A at the end. I have a feeling that a lot more knowledge would have been gained by having members of the audience riff along with the panelists instead of the traditional "wisdom from above" model used in thuis panel.

An additional meta-comment about the conference is that so far I've been unable to get into 2 out of the 3 sessions I wanted to attend this morning because they were filled to overflowing. Given how much folks are paying for the conference and how much experience the O'Reilly folks have with holding conferences, one wouldn't see such problems occur.

Brent Simmons has confirmed that Newsgator has purchased NetNewsWire. One of the reasons Brent gave for the purchase is excerpted below

The first is that we get requests constantly about syncing—not just better syncing, not just between copies of NetNewsWire, but with Windows RSS readers, PDAs, Outlook, and so on. People even ask us to create a website version for when they’re away from their normal computers.

We couldn’t do all this on our own—but we agree completely with NetNewsWire users who tell us that RSS is hugely important, too important to have to read the same news items twice on different computers and different devices.

NewsGator was already working on this—but they didn’t have a Mac client. It was almost like putting together a jigsaw puzzle: NetNewsWire fit right in!

Users of RSS Bandit already have access to a robust syncing mechanism that allows users to use the application on multiple computers without having to mark the same item as read twice. However there are several limitations to the mechanisms used by RSS Bandit today.

• The user has to set up a server (FTP or WebDAV)
• The user cannot synchronize with an online reader for the times they aren't at a PC with RSS Bandit installed
• The user cannot synchronize with a Mac-based reader for the times they are using a Mac and not a PC

I have been hit by all three of these limitations at one time or the other this year. All three of these problems will be a thing of the past with the Nightcrawler release of RSS Bandit which will support synchronization with Newsgator Online via the Newsgator API.

Congratulations to Brent and Greg on their union. And congratulations to RSS Bandit users who will now be able to sync between NetNewsWire on the Mac and RSS Bandit on the PC.

I took a look at Ning today and I find it interesting from a number of perspectives. On the one hand, it's simply a toolkit for building websites. The same kind of toolkit you see hawked on infomercials while channel surfing at midnight because you have insomnia. The main difference is that instead of a CD that contains software for building eCommerce websites it's an online website for building 'social software' websites. On the other hand, it shows an interesting direction for web platforms to move in.

Let's say you are Yahoo! or MSN and you now have an extensive array of web services at http://developer.yahoo.com or http://msdn.microsoft.com/msn. So now developers can build desktop applications or their own websites that interact with your APIs. These are akin to the low level APIs for interacting with your web platform. The next step is to build the equivalent of rich, high-level components for interacting with your services so people don't have to mess with the low level details. What I find interesting about Ning isn't "Here's another place where I can build yet another social network different from my IM buddy list/Orkut/Yahoo! 360/Friendster/etc clique" but instead "Here's a place where I can build new apps that harness my existing social network from my IM buddy list/Orkut/Yahoo! 360/Friendster/etc". That would be pretty revolutionary if the various parties involved would be interested in opening their social networks to such mashups.

I suspect I'll be trying to track down some folks from Ning while at the Web 2.0 conference this week.

Anyway, gotta go catch my flight.

I've been reading the Mini-Microsoft blog for a couple of months now and recently considered unsubscribing. The main reason I haven't is that the recent story in Business Week about the blog has attracted a lot of readers which makes the comment threads interesting in a Slashdot kinda way. Since I couldn't locate an email address for the blog's author on the front page of the blog, I'll just post my comments here on why the blog jumped the shark for me.

1. Complaints about Symptoms instead of the Root Problems: A lot of the things complained about by the author of the Mini-Microsoft blog are symptoms of a single problem. About six years ago, Motley Fool ran an article entitled The 12 Simple Secrets of Microsoft Management which listed a number of characteristics of the company which made it successfull. The fourth item on the list is Require Failure and it states

   In contrast, at Microsoft, failure is expected, and even required because risking failure is the only way to push the envelope. As a result, Microsofties relentlessly pursue success without fear of failure. And if they fail, they understand that the key is to fail quickly and not waste time.

   One of the unfortunate things about a culture that turns a blind eye to failure is that this eventually there is no difference between requiring failure and a lack of accountability. A lot of the things Mini complains about point to an environment where  a lack of accountability runs rampant. I'd rather see him ring the bell about these issues [which he does every once in a while] as opposed to meaningless distractions like complaining about vague ship dates or asking for mass firings because the company is "too large".

2. Microsoft is Lots of Little Companies: One thing that isn't clear from Mini's posts is that a number of the complaints he raises are organization specific. The culture in the Office group is different from that at MSN, the issues facing the people working in the Windows group are different from those facing the folks working on XBox. Mini's blog not only isn't representative but it doesn't seem like he pays much detail to what is happening outside of his group. For example, it is quite telling that he didn't know that the ship date for Visual Studio 2005 was announced a while ago. Given how many people within the company work on and are impacted by the shipping of Whidbey & Yukon, it seems clear that Mini doesn't pay much attention to what is going on outside of his group.

3. Stack Ranking: This point is a probably a repeat of my first one. First of all when it comes to performance reviews, I tend to agree with Joel Spolsky that Incentive Pay Considered Harmful. Joel wrote

   And herein lies the rub. Most people think that they do pretty good work (even if they don't). It's just a little trick our minds play on us to keep life bearable. So if everybody thinks they do good work, and the reviews are merely correct (which is not very easy to achieve), then most people will be disappointed by their reviews. The cost of this in morale is hard to understate. On teams where performance reviews are done honestly, they tend to result in a week or so of depressed morale, moping, and some resignations. They tend to drive wedges between team members, often because the poorly-rated are jealous of the highly-rated, in a process that DeMarco and Lister call teamicide: the inadvertent destruction of jelled teams.

   In general, systems where you try to competitively rank people and reward them based on their rankings suck. A lot. When you combine this with the current rewards associated with positive rankings at Microsoft, then you have a system that doubly sucks. I think Mini gets this but he keeps talking about alternative performance review systems even though there is lots of evidence that incentive pay systems simply do not work.

Those are the top three reasons that I find myself losing interest in keeping up with the Mini-Microsoft blog. However I'll probably keep reading because the comments now have me gawking at them regularly, sometimes even more than I do at Slashdot.

Tim O'Reilly has posted What Is Web 2.0? : Design Patterns and Business Models for the Next Generation of Software which further convinced me that the definition of Web 2.0 used by Tim O'Reilly and his ilk may be too wide to be useful. In the conclusion of his article he writes

Core Competencies of Web 2.0 Companies

In exploring the seven principles above, we've highlighted some of the principal features of Web 2.0. Each of the examples we've explored demonstrates one or more of those key principles, but may miss others. Let's close, therefore, by summarizing what we believe to be the core competencies of Web 2.0 companies:

• Services, not packaged software, with cost-effective scalability
• Control over unique, hard-to-recreate data sources that get richer as more people use them
• Trusting users as co-developers
• Harnessing collective intelligence
• Leveraging the long tail through customer self-service
• Software above the level of a single device
• Lightweight user interfaces, development models, AND business models

The next time a company claims that it's "Web 2.0," test their features against the list above.

The list seems redundant in some places and could probably be reduced to 3 points. Half the bullet points all seem to say that the company should expose Web services [in this context I mean services over the Web whether they be SOAP, REST, POX/HTTP, RSS, etc]. So that's point number one. The second key idea seems to be that of harnessing collective intelligence such as with Amazon's recommendation engine, Wikipedia entries and folksonomies/tagging systems. The final key concept is that Web 2.0 companies leverage the long tail. One example of the difference between Web 1.0 and Web 2.0 when it comes to harnessing the long tail is the difference between http://www.msn.com which is a portal that has news and information of general interest that aims at appealing to broad audiences (one size fits all) and http://www.start.com which encourages people to build their own portal that fits their needs (every niche is king).

So let's review. Tim O'Reilly's essay can be reduced to the following litmus test for whether an offering is Web 2.0 or not

• Exposes Web services that can be accessed on any device or platform by any developer or user. RSS feeds, RESTful APIs and SOAP APIs are all examples of Web services.
• Harnesses the collective intelligence knowledge of its user base to benefit users
• Leverages the long tail through customer self-service

So using either Tim O'Reilly's list or mine, I'd be curious to see how many people think http://www.myspace.com is a Web 2.0 offerings or not. If not, why not? If so, please tell me why you think all the folks who've called MySpace a Web 2.0 offering are wrong in my comments. For the record, I think it isn't but would like to compare my reasons with those of other people out there.

Brian Jones has a post entitled Native PDF support in Office "12" where he writes

Today's another exciting day as we move closer to Beta 1. We are just wrapping up the MVP summit here in Redmond and we've finally announced another piece of functionality I've wanted to talk about for a long time now. This afternoon Steven Sinofsky announced to our MVPs that we will build in native support for the PDF format in Office "12".  I constantly get asked by customers if we can build in this support for publishing documents as PDF files, and now I can thankfully say "yes!" It's something we've been hearing about for years, and earlier in this project we decided that while there were already existing third party tools for doing this, we should do the work to build the functionality natively into the product.

The PDF support will be built into Word, Excel, PowerPoint, Access, Publisher, OneNote, Visio, and InfoPath! I love how well this new functionality will work in combination with the new Open XML formats in Word, Excel, and PowerPoint. We've really heard the feedback that sharing documents across multiple platforms and long term archiving are really important. People now have a couple options here, with the existing support for HTML and RTF, and now the new support for Open XML formats and PDF!

This is a very welcome surprise. The Office team is one of the few groups on main campus who seem to consistently get it. Of course, the first thought that crossed my mind was the one asked in the second comment in response to Brian's post.

There's been a bunch of MSN Virtual Earth hacking going on in my building over the past couple of weeks. There was my Seattle Movie Finder hack. Recently two folks on the MSN Messenger team created a shared map browsing application using the recently released MSN Messenger Activity API. Chandu Thota has the details in his post Virtual Earth and MSN Messenger : Peer-2-Peer Mapping Experience

If you are running MSN Messenger 6.0 or higher, open a conversation with your contact and click on "Activities" menu item; it will display a list of activities that you can use which includes "Virtual Earth Shared Map" as shown below:

Once you and your contact accept this activity you both can find, pan and zoom on the Virtual Earth map all interactively, like the one shown below:

Okay, I don't want to waste your time anymore - this is one of the coolest things I have seen in this space - try it out! you won't be disappointed! :)

PS: Kudos to Steve Gordon and Shree Madhavapeddi from MSN for creating such a wonderful app!

I got a demo of this from Steve and Shree last week, I didn't realize that it would show up in the MSN Messenger application so soon. That is some quick turn around time.

I also got a demo of a cool Start.com gadget which uses MSN Virtual Earth from Matt this week. I wonder how long that will take to sneak out onto the Web.

PS: In his post about this Robert Scoble states that the application was created by Scott Swanson. This isn't accurate, Scott wrote a similar application as a PDC demo but the version you can get in MSN Messenger activities menu isn't it.

There have been a number of amusing discussions in the recent back and forth between Robert Scoble and several others on whether OPML is a crappy XML format. In posts such as OPML "crappy" Robertson says and More on crappy formats Robert defends OPML. I've seen some really poor arguments made as people rushed to bash Dave Winer and OPML but  none made me want to join the discussion until this morning.

In the post Some one has to say it again… brainwagon writes

Take for example Mark Pilgrim's comments:

I just tested the 59 RSS feeds I subscribe to in my news aggregator; 5 were not well-formed XML. 2 of these were due to unescaped ampersands; 2 were illegal high-bit characters; and then there's The Register (RSS), which publishes a feed with such a wide variety of problems that it's typically well-formed only two days each month. (I actually tracked it for a month once to test this. 28 days off; 2 days on.) I also just tested the 100 most recently updated RSS feeds listed on blo.gs (a weblog tracking site); 14 were not well-formed XML.

The reason just isn't that programmers are lazy (we are, but we also like stuff to work). The fact is that the specification itself is ambiguous and weak enough that nobody really knows what it means. As a result, there are all sorts of flavors of RSS out there, and parsing them is a big hassle.

The promise of XML was that you could ignore the format and manipulate data using standard off-the-shelf-tools. But that promise is largely negated by the ambiguity in the specification, which results in ill-formed RSS feeds, which cannot be parsed by standard XML feeds. Since Dave Winer himself managed to get it wrong as late as the date of the above article (probably due to an error that I myself have done, cutting and pasting unsafe text into Wordpress) we really can't say that it's because people don't understand the specification unless we are willing to state that Dave himself doesn't understand the specification.

As someone who has (i) written a moderately popular RSS reader and (ii) worked on the XML team at Microsoft for three years, I know a thing or two about XML-related specifications. Blaming malformed XML in RSS feeds on the RSS specification is silly. That's like blaming the large number of HTML pages that don't validate on the W3C's HTML specification instead of on the fact that instead of erroring on invalid web pages web browsers actually try to render them. If web browsers didn't render invalid web pages then they wouldn't exist on the Web.

Similarly, if every aggregator rejected invalid feeds then they wouldn't exist. However, just like in the browser wars, aggregator authors consider it a competitive advantage to be able to handle malformed feeds. This has nothing to do with the quality of the RSS specification [or the HTML specification], this is all about applications trying to get marketshare.  

As for whether OPML is a crappy spec? I've had to read a lot of technology specifications in my day from W3C recommendations and IETF RFCs to API documentation and informal specs. They all suck in their own ways. However experience has thought me that the bigger the spec, the more it sucks. Given that, I'd rather have a short, human readable spec that sucks a little (e.g. RSS, XML-RPC, OPML etc.) than a large, jargon filled specificaton which sucks a whole lot more (e.g. WSDL, XML Schema, C++, etc). Then there's the issue of using the right tool for the job but I'll leave that rant for another day.

While using Firefox this morning, I just realized something was missing. There is a Google Toolbar for Firefox, there is a Yahoo! Toolbar for Firefox, so how come there isn't an MSN Toolbar for Firefox? Just yesterday, Ken Moss who runs the MSN Search team posted on their blog about MSN Search Plugins for Firefox where he wrote

However, some of our customers prefer using Firefox and we respect that choice.  Some developers in our user community have created Firefox plug-ins to make it easy to do searches on MSN from the Firefox search box.  Even though it’s currently buried in Firefox under “Add Engines… Find lots of other search engines…”, it seems that our customers have been finding it since we’re listed as one of the most popular search engine plugins.

I use Firefox sometimes in the course of my job – and when I do, I love having the MSN Search engine plugged-in up in the chrome.  If you’re currently a Firefox user – I hope you’ll enjoy this little nugget. For more MSN Search fun with Firefox (or IE!), try out the PDC version of MSN Search enabled by a Trixie / Greasemonkey script.

It's cool to see the MSN Search team giving a shout out to plugins built by the developer community but I think it would be even cooler if we step up to the plate like Yahoo! and Google have done by providing an official,  full fledged toolbar for Firefox.

Kitty came by my office to remind me that the Web 2.0 conference is next week. As part of the lead up to the conference I can see the technology geek blogosphere is buzzing with the What is Web 2.0? discussion which was sparked off by Tim O'Reilly's posting of the Web 2.0 meme map created during FooCamp. The meme map is below for the few folks who haven't seen it 

The meme map is visual indication that "Web 2.0" has joined "SOA" as a buzzword that is too ill-defined to have a serious technical discussion about. It is now associated with every hip trend on the Web. Social Networking? That's Web 2.0. Websites with APIs? That's Web 2.0. The Long Tail? That's Web 2.0. AJAX? That's Web 2.0. Tagging and Folksonomies? That's Web 2.0 too. Even blogging? Yep, Web 2.0.

I think the idea and trend towards the 'Web as a platform' is an important one and I find it unfortunate that the discussion is being muddied by hypesters who are trying to fill seats in conference rooms and sell books.

I'm in the process of updating my Billl Gates Thinkweek paper on MSN and Web platforms to account for the fact that some of my recommendations are now a reality (I helped launch http://msdn.microsoft.com/msn) and more importantly given recent developments it needs to change tone from a call to action to being more prescriptive. One of the things I'm considering is removing references to "Web 2.0" in the paper given that it may cause a bozo bit to be flipped. What do you think?