In a blog post entitled Reinventing HTML Tim Berners-Lee writes

The perceived accountability of the HTML group has been an issue. Sometimes this was a departure from the W3C process, sometimes a sticking to it in principle, but not actually providing assurances to commenters. An issue was the formation of the breakaway WHAT WG, which attracted reviewers though it did not have a process or specific accountability measures itself.

There has been discussion in blogs where Daniel Glazman, Björn Hörmann, Molly Holzschlag, Eric Meyer, and Jeffrey Zeldman and others have shared concerns about W3C works particularly in the HTML area. The validator and other subjects cropped up too, but let's focus on HTML now. We had a W3C retreat in which we discussed what to do about these things.

Some things are very clear. It is really important to have real developers on the ground involved with the development of HTML. It is also really important to have browser makers intimately involved and committed. And also all the other stakeholders, including users and user companies and makers of related products.

Some things are clearer with hindsight of several years. It is necessary to evolve HTML incrementally. The attempt to get the world to switch to XML, including quotes around attribute values and slashes in empty tags and namespaces all at once didn't work. The large HTML-generating public did not move, largely because the browsers didn't complain. Some large communities did shift and are enjoying the fruits of well-formed systems, but not all. It is important to maintain HTML incrementally, as well as continuing a transition to well-formed world, and developing more power in that world.

The plan is to charter a completely new HTML group. Unlike the previous one, this one will be chartered to do incremental improvements to HTML, as also in parallel xHTML. It will have a different chair and staff contact. It will work on HTML and xHTML together. We have strong support for this group, from many people we have talked to, including browser makers.

Wow. It's good to see the W3C reacting to all the negative criticism it has received on its stewardship of HTML in recent times. A few months ago I linked to a number of the complaints from the markup geek crowd that  Tim Berners-Lee references in my post entitled W3C Process is Broken? Film at 11. Although it was clear the writing was on the wall, I didn't expect the W3C to change its course anytime soon. The inertia within that  organization is just that massive. With browser makers and Web developers being disenchanted with the W3C, this is the only thing they could do if they planned to remain relevant in the world of Web standards. Kudos to TimBL and the rest of the W3C crew for making this course correction. 

PS: I really need a personalized meme tracker. The linked post didn't make it onto TechMeme but it did make it onto the meme tracker on Planet Interwingly. I suspect it would have made it onto my list of 'interesting posts' if I had a personalized meme tracker running over my feed list as well.

 

I read an interesting pair of posts about Web startups competing with big companies like Google and Microsoft over the weekend. The first was a post by Bill Burnham entitled Search Applications: Search Startups Are Dead, Long Live Search Startups where he writes 

In response to a question about the prospects for the myriad of search start-ups looking for funding Peter basically said, and I am paraphrasing somewhat, that search start-ups, in the vein of Google, Yahoo Ask, etc. are dead.  Not because search isn’t a great place to be or because they can’t create innovative technologies, but because the investment required to build and operate an Internet-scale, high performance crawling, indexing, and query serving farm were now so great that only the largest Internet companies had a chance of competing.

Priced Out of the Market
While the comment might strike some as self-serving, the fact of the matter is that it is true.  Any start-up trying to displace Google, Yahoo, or even MSN or Ask (or for that matter any VC trying to fund them) should just get in their car (or hop on a plane) and go look at Google’s new server farms at The Dales in Oregon.  And if that doesn’t convince them they should head up the Columbia river a bit and check out Microsoft and Yahoo’s digs.   The costs to compete in core search , are now simply to high.

Bill then goes on to argue that the opportunity may lay in building applications on top of the APIs provided by the big software companies such as the Alexa Web Search Platform instead of trying to compete head to head with companies that have already invested hundreds of millions of dollars in building out their search infrastructure. In a post in response to Bill Burnham, Tim O'Reilly agrees that "we're entering the platform phase of Web 2.0, in which first generation applications are going to turn into platforms".

Dave Winer comes to the same realization but with a different conclusion in his post Someday search will be old too where he writes

Many years ago, when the Internet was still the domain of geeks, researchers and college students, the smart folks often said that the opportunities for new software companies were over, it simply required too much scale to compete in an industry dominated by Lotus, Microsoft and Ashton-Tate. Now it's clear how ridiculous that was, even though it was correct. The next layer comes on not by building on the old layer (a trick, the guy you're building on will eat your lunch), or re-doing what they did (what the naysayers correctly say you can't do), but by starting from a different place and building something new, and so different that the old guys don't understand it and don't feel threatened by it.

Dave seems to be disagreeing with Bill Burnham and Tim O'Reilly that building on the platforms provided by the big software companies is a good idea because the companies can turn around and compete with you. Although I can see Dave's point, simply having the same base platform doesn't mean you can build the same application. We all have access to Linux, Apache, MySQL and PHP but how many people have or can build something like Flickr? You should definitely examine the risks when building on any platform but eventually you have to be ready to face off against a competitor that may have access to the same or a similar platform as you. You're development platform cannot be your only differentiator.

I do agree more with the spirit of what Dave Winer is recommending than what Tim O'Reilly  & Bill Burnham are. If I ever founded a startup, it would take advantage of the knowledge I have of what big companies like Microsoft are good at and what they aren't. For example, YouTube would never have come out of Microsoft because the company would have been too scared of lawsuits. Of course, now that YouTube has money they are getting hit up from every angle so that fear does make sense for a big company but not a small startup especially if the exit strategy is to flip it. 

 

I had some DNS issues last week because I forgot to renew my domain name registration. According to my FeedBurner statistics page, one of the consequences of this lapse on my part is that I've lost about 1,000 subscribers to my feed on Rojo. It looks like they may have decommissioned my feed in their service once the domain name stopped resolving after a couple of hours. This seems like a reasonable thing to do.

I've sent some mail to the Rojo folks through their feedback email address but don't hold out much for ever getting a response. It may be that there is nothing for them to fix since the relationship between my feed and the subscribers may have been deleted from their database once my domain was considered to be gone.

Anyway, if you are a Rojo user wondering why my feed disappeared from you subscription list, now you know. 

 

Thanks to Marc Canter's post, And what about the Aggregator Vendors? I'm reminded that there is still work to do before I've turned RSS Bandit into a satisfactory podcast client. With the basic code for downloading podcasts now implemented I've now turned my mind to how to get the downloaded rich media content onto a user's device and/or media player of choice. For the most part, I'm assuming that people are either using iTunes (version 6 or later) or Windows Media Player (version 10 or later).

Getting the music or videos onto a playlist in the users media player seems to be mostly straightforward. I've found C# sample code for creating playlists in iTunes and creating playlists in Windows Media Player. The tricky part has been figuring out how to get the music onto the users portable media player  (e.g. their iPod or Creative Zen) as opposed to their PC hard drive and creating a playlist on the device that includes the newly downloaded file. Doing the first isn't that bad if the device can be mounted as an external hard drive because the user can just specify that RSS Bandit should download files to a folder on that external drive. On the other hand, none of the sample code I've seen talks about creating a playlist on the portable media player. Any help here from my readers would be greatly appreciated. 

PS: I briefly considered embedding an instance of Windows Media Player in RSS Bandit but that seems like overkill compared to just having a [Play] button that opens the file in the user's preferred media player for that file type.

 

Robert Scoble has a blog post entitled New audience metric needed: engagement where he writes

I was just reading Jeneane Sessum’s post about the latest Ze Frank/Rocketboom dustup and she’s right, we need to measure stuff other than just whether a download got completed or not. She says we need a “likeability” stat. I think it goes further than that.

There’s another stat out there called “engagement.” No one is measuring it that I know of. What do I mean?

Well, I’ve compared notes with several bloggers and journalists and when the Register links to us we get almost no traffic. But they claim to have millions of readers. So, if millions of people are hanging out there but no one is willing to click a link, that means their audience has low engagement. The Register is among the lowest that I can see.

Compare that to Digg. How many people hang out there every day? Maybe a million, but probably less. Yet if you get linked to from Digg you’ll see 30,000 to 60,000 people show up. And these people don’t just read. They get involved. I can tell when Digg links to me cause the comments for that post go up too.

I've heard Frank Shaw state anecdotally that blogs are more 'influential' than traditional media websites. You get more click-throughs from being mentioned in a popular blog than from being mentioned in a more popular technology website. I'm interested in theories on why this is the case. Could it be that bloggers are more 'influential' over their audience than traditional media? Are blog readers more 'engaged' as Robert puts it?

PS: I read Jeneane's Sessum's post to be quite irritating. The smug assumption that if you like something then it must be more popular or at least 'better' on some made-up axis than what everyone else likes is a hallmark of the blogosphere echo-chamber. You see the same kind of egotistical thinking in Stowe Boyd's post criticizing Yahoo! bookmarks in comparison to del.icio.us.

 

In his blog post entitled Yahoo Bookmarks Enters 21st Century Mike Arrington writes

Yahoo is unveiling an entirely new Bookmarks product this evening at new.bookmarks.yahoo.com - new interface, new back-end, the works. A screencast created by Yahoo developer Tom Chi is here which gives an excellent overview of the service (Chi also created the background music for the screencast). Compare that to the existing Bookmarks product (screenshot is here) and it’s clear how significant the overhaul is.

Yahoo Bookmarks, while invisible to most cutting edge web users, still claims around 20 million active users (compared to only 1 million for del.icio.us).

Until today, Yahoo Bookmarks (which is a separate product from del.icio.us and My Web) stored only the URL, title and comment for a particular bookmark. The new product caches all text on the page, stores a thumbnail view, and allows both categorization (folders) and tagging of each bookmark.

There are two things that I found striking about this announcement. The first is that Yahoo! has three bookmarking products; del.icio.us, My Web and Yahoo! Bookmarks. The second is that it looks like my shunning del.icio.us while continuing to use Yahoo! Bookmarks isn't so weird after all, there are twenty times more people using Yahoo's regular bookmarking service compared to its social bookmarking product.

At this rate it looks like del.icio.us is destined to be a niche service unless something radical is done such as merging Yahoo! Bookmarks and del.icio.us into a single service. Of course, given the user outcry when Netscape.com revamped to be more Digg-like, the folks at Yahoo! may not have the stomach for this.

Faint heart never won fair lady. 

 

In the past few months there have been a couple of announcements from the big search engines such as Yahoo! and Live Search on the topic of enabling people to build their own custom search engines. Google has finally showed up at the party with their own offering which was unveiled today. Below are my thoughts on their offering versus that of Windows Live.

Google Co-op

In his blog post entitled Review: Custom Search Engine Matt Cutts of Google writes

Google just announced something that I’m really jazzed about: Google Custom Search Engine. Several people mentioned that Google’s Accessible Search was built by using Google Co-op under the hood. Co-op has opened much of that power up to the public, so that anyone can build a custom search engine.

Most custom search engines (whether it be Google’s free sitesearch or Yahoo! Search Builder) only let you select one site to search, or you can offer websearch. Even Rollyo only lets you search over 25 sites.

This new offering lets you easily add hundreds (thousands?) of urls. You can search over ONLY the sites you choose, or (my favorite) you can apply a boost to the sites you choose, with regular websearch as a backfill. That’s really nice, because if your chosen urls talk about a subject, you’ll often get matches from those urls, but if the user types something completely unrelated, you’ll still get web results back. So it’s a true custom search engine, not just an engine restricted to showing matches from some domains.

You can also choose to exclude results from different sites. As far as I can tell, this happens in pretty close to real-time, even for complex url patterns. For example, I added the pattern “google.com/*” and started to get results from the Google directory, so I excluded “google.com/Top/*” and the Google directory results went away immediately.

There is also a screenshot included in Mike Arrington's post at TechCrunch entitled Google Co-op Launches which is excerpted below

This isn’t new - Rollyo, Eurekster and Yahoo already have similar products. But Google is also offering, as an option, to bundle the service with Google Adsense ads and share revenue with websites that embed the custom search engine into their site. Only Eurekster currently shares revenue with users. Yahoo’s product, which got a lot of press at launch, has barely been mentioned in the nearly three months since then.

I didn't even realize that Yahoo! had an offering in this space until reading the TechCrunch entry. This doesn't seem to have gotten that much blogosphere love.

Live Search Macros

The Windows Live Search team wrote about the changes to the Search Macros feature originally announced in March in their blog post entitled Create your own search engine (an update to Live Search Macros) which states

Search Macros are personalized search engines for any topic area of interest.  You can create them, use them, share them with friends or discover macros created by the community on Windows Live Gallery.

I’d like to use this post to give you a basic overview of using and creating macros.  We’ll use future posts to dive into more of the nitty gritty on specific macros features.

Finding and using macros

Users of the first Macros release told us that using a macro was difficult and not very user friendly.  In this release, every macro now has its own homepage and human readable URL.  This makes them much easier to use, bookmark, and send to friends over email or IM.  For example, check out the homepage for the Reference Sites Search Engine macro (at http://search.live.com/macros/livesearch/reference):
...
Enter a search term on this page and press Enter. You’ll be taken to the main Live Search page to see your results.
...
On the results page you'll see that the macro’s name appears in the search bar at the top of the screen.  This enables you to switch back and forth between Web, Images, Local, QnA and your favorite macros.

Here are some macros to try:

•   Search top reference site on the Web with the Reference Sites Search Engine
•   Search all content on the Microsoft Developer Network with the MSDN Search Engine
•   Search all of the latest video game information with the Video Games Search Engine

You can also find many more in the Windows Live Gallery!

Andy Edmonds from the Live Search team has written a couple of blog posts about the cool things you can do with search macros such as Search Macros Recap: DiggRank and the Power of Trusted Networks

Macros are often compared to Rollyo or other site bundling search offerings, but I hope the blog post describing LinkFromDomain, LinkDomain, and featuring other operators, sets the record straight.  Defining a set of sites to search is cool, and an idea well due to be commonly available. To be fair, Rollyo's UI and integration is slick, but  Micah Alpern hacked up a search of his blog, the blogs he linked to, or the web at large with a Google API hack back in 2003!

Using the link domain operators, you can go well beyond a simple set of sites.  You can:

• keep a living list of the sites that link to you and search them
• keep a living list of the sites you link to and search them
• do the same for a set of trusted sites

Access to other advanced syntax differentiates further from simple site search amalgamations.  Heck, Scoble pontificated about a search engine that excluded blogs that participate in pay per post.  While I didn't figure out a way to focus this on only those PPP bloggers who don't disclose their interest, I think it's impressive that the basics can be done at all.  It's called macro:andyed.realBloggers, and uses -inbody:counttrackula.com to exclude sites that use the PPP tracking script (I think!) and hasfeed: to restrict to blogs (or other pages with syndication).

Super Hubs: DiggRank
The promise of personal networks of trust in information retrieval is not fully realized by the macros offering, but it's an important step in the right direction.  For super-hubs, like Digg or Delicious, linkFromDomain captures some really interesting human attentional residue.

Let me introduce macro:andyed.DiggRank. Try it for:

• coffee
• George Bush
• midterm elections

The Bottom Line

I tried out both services as well as Yahoo! Search Builder and they all seem to have some room for improvement. Both Google & Yahoo! have primarily built a way to add a custom search box for your site. Windows Live Search is primarily about adding your own customized search results to your search engine of choice. I think both scenarios should be covered by all the services. I think Live Search should give me the option of adding a search box on my blog that is powered by a search macro I wrote. Similarly, I'd like to be able to perform custom searches from the Google or Yahoo! search UI without having to remember how to get to http://google.com/coop/cse/ or http://builder.search.yahoo.com/. I have to agree with Sergey Brin here, Features, Not Products. This is yet another Google service that I have to perform a Google search for before I can find it and use it (others are Google Music Search and Google Blog Search).

One thing I do like about Google and Yahoo!'s options is that they provide a more user friendly UI for creating complex searches than Live Search which provides you with direct access to the search operators. This is more powerful and desirable to geeks like me but it is not very user friendly for the non-geek. A checkbox with 'prefer search results from these sites' is preferable to crafting a search query with "prefer:http://www.25hoursaday.com AND prefer:http://www.rssbandit.org".

The management page for Google Co-op needs a lot of work. It is sparse in the typical Google way but it also doesn't seem coherent nor does it give you enough information about what you can or should be doing. The management page for Yahoo! Search Builder is a lot more coherently organized and aesthetically pleasing. The search macro management page for Live Search also could do with some improvement, primarily in simplifying the process for creating complex macros.

PS: Revenue sharing is a nice touch by Google and I'd be quite surprised if Yahoo! doesn't follow suite soon.

 

I found the time to start on integrating the ability to download podcasts in RSS Bandit today. One of the things I realized is that we can't make the blanket assumption that every enclosure we see in an RSS or Atom feed is a podcast. I decided that instead of only exposting enclosures as podcasts, it would be more generic and maybe more useful to treat them as file attachments to feed items.

Below is a screenshot of what this looks like

There are other bits of UI we have to add such as a download queue and a podcasts inbox. I've also been thinking of adding the option to enqueue downloaded podcasts in iTunes or Windows Media Player. I wonder if the Zune player will provide an API so I can add that as an option as well. I should ask around at  work.

I also explicitly decided against any sort of podcasts directory functionality. If we don't typically use 'blog directories' why do we need feed directories? We're about two weeks from being done with this feature after which we'll ship a beta of the Jubilee release. My current thinking is a beta before Thanksgiving in the U.S. and the final release just before the Christmas holidays.

 

I got an email over the weekend from a friend of mine who's leaving Microsoft. I wasn't surprised to see him leave Microsoft, given that the every project he's worked on at Microsoft has either been cancelled mid-project or end of lifed in that release. After being at Microsoft for five years, I've now begun to see the signs that a project is likely to crash and burn early on. Below is a top five list of signs your software project is in trouble I compiled as part of my 'parting career advice' to my intern.

1. Schedule Chicken: This is typically a sign that the project's schedules are unrealistic. A project with unrealistic schedule is either an indication of poor communication between layers in the product team or even worse, bad management that punishes the messenger when there is bad news (e.g. poor initial estimation of project length). The main problem with schedule chicken is that you can be "date driven" or you can be "quality driven", you can't be both.

2. Scope Creep: Requirements changing as a software project progresses are natural. They can change due to feedback from the customer after they get to try out a prototype, due to changes in the competitive landscape or because the original requirements had hidden conditions which were not discovered until after implementation. When things get bad is when the goals of a project are changed or increased significantly without a corresponding significant change to the expected timeframe for delivery. WinFS merging with Object Spaces is my canonical example of scope creep at Microsoft.

3. Underresourced: You don't bring a knife to a gun fight. So you shouldn't expect that 3 developers and $50,000 will be able to compete with the Googles and Microsofts of the world. Similarly, if you work at a big company and you have a handful of folks working on a product where competitors have large teams or entire companies working on the same problem space, you're probably in over your head.  

4. Second System Syndrome: Once you ship a software application, it instantly becomes legacy code. To a developer this means there is something newer and sexier that can solve the same problem in a more elegant way. Eventually a project is started which is intended to replace the existing product which customers are finding useful. This is often a double whammy. The new project is hamstrung out of the gate by having to meet customer expectations on backwards compatibility, performance and new features in comparison to the old product. This burden is often a crushing weight on the second system which eventually collapses under the strain. In addition, the old project is often abandoned or at best put in "maintenance mode" with only a skeleton crew working on it even though it pays the bills.  

5. No Entrance Strategy: There is a lot of talk in the software industry of exit strategies but a lot of the time software products do not have an entrance strategy. How do you get people to use the application? How do you get the first 100,000 or 1 million users? Sometimes in big companies, there is also the corporate strategy tax to consider when deciding whether a product has an entrance strategy or not. When I was on the XML team at Microsoft, there were folks on the team working on a project they called X#. The project was basically C# with extensions to handle relational and XML data access as operations native to the programming language. I attended an internal presentation about the project and when asked what the deliverable from the project would be, the team members actually showed a Photoshoped image of a Microsoft Visual C# box which read Microsoft Visual X#. Of course, this was at the time Microsoft was taking heat for introducing both C# & Visual Basic.NET at the same time. It was unlikely that Microsoft would ship a third similar language anytime soon. The project was killed, resurrected  and morphed a couple of times. The story eventually ended happily with a lot of the innovations in the language eventually showing up as .NET Language Integrated Query (LINQ) (aka C# 3.0). That was one instance with a happy ending, a counter example is the new file system for Windows being cancelled a few years after it was announced to be shipping separately from the operating system. A file system that doesn't ship with the operating system doesn't sound like a product with an entrance strategy to me. How about you?

 

Jeffrey Zeldman has a blog post entitled Web 2.0 Thinking Game where he writes

A few weeks back, The Economist was calling “Web 2.0″ a trend. Their phrase was, “hot Web 2.0 trend.” The magazine now intends “Web 2.0″ to be understood as a sort of second edition:

This week’s pairing of Google and YouTube may come to be remembered as the moment “Web 2.0″—ie, the web, version two—came of age.

Clearly “Web 2.0″ means different things to different journalists on different days. Mostly it means nothing—except a bigger paycheck. But let’s simplify what The Economist is saying:

Web 1.0: AOL buys Time Warner.
Web 2.0: Google buys YouTube.

Put another way:

Web 1.0: New media company buys old media company.
Web 2.0: New media company buys new media company.

If we’re stuck with this meaningless Web 2.0 label, let’s at least have some fun with it. Here’s my new game. I’ll start, you finish:
...
Web 1.0: Users create the content (Slashdot).
Web 2.0: Users create the content (Flickr).

Web 1.0: Crap sites on Geocities.
Web 2.0: Crap sites on MySpace.
...
Web 1.0: Karma Points.
Web 2.0: Diggs.

Web 1.0: Cool Site of the Day.
Web 2.0: Technorati.com.
...
Now you try it!

There are a lot of funny ones in the comments as well such as

Web 1.0: Old folks have no clue.
Web 2.0: My parents just left a comment on my blog. - Charlie

Web 1.0: Rational Unified Process implementing J2EE
Web 2.0: “Getting Real” using RoR - Kevan Emmot

Web 1.0: 20,000 Hits on my webpage!
Web 2.0: Ive been dugg 1000 times on Digg! - Regnard Raquedan

Web 1.0: Pamela Anderson
Web 2.0: Paris Hilton - Chase

Here're a couple of my own, let's see what you guys come up with

1. Web 1.0: Netscape IPO
   Web 2.0: Google IPO
   

2. Web 1.0: My startup just IPOed, I'm gonna be rich
   Web 2.0: My startup just got bought by Google, I'm gonna be rich

3. Web 1.0: Napster
   Web 2.0: YouTube
   

4. Web 1.0: Yahoo! Bookmarks
   Web 2.0: del.icio.us
   

5. Web 1.0: Java applets
   Web 2.0: Widgets
   

6. Web 1.0: Beth Goza
   Web 2.0: Niniane Wang