Julien Couvreur has a blog post entitled Web API authentication for mashups where he talks about authentication and Web APIs. This is a topic that is near and dear to my heart since getting this right is very important for the Windows Live developer platform. Julien writes

Authorization techniques:

A number of techniques for controlling access to web APIs are generally used: user authentication cookies, API keys and crossdomain policy files. The problem is that API keys and crossdomain policy files are too restrictive because the service needs to decide which third-parties to let in.

On the other end, access control based on the user authentication cookies are very open to un-planned integration, but also create a huge phishing risk. This is a classic example of the confused deputy problems that appear in principal-based security models.

As a result, most web APIs today don't involve any user data (search, maps, ...) or non sensitive user data.

Yahoo APIs:

Yahoo appears to be tackling the challenge with its announced "browser-based authentication". From the little information I could gather so far, it seems less of an authentication than an authorization system. Unlike cookie based approaches, which give access to any agent presenting user credentials (principal-based security), it appears to follow a capability-based security model, which only grants access if the agent uses the proper "secure handle" or "capability" to call the service. Such capabilities are sufficient to gain access to the service and don't need any additional authentication, they are communicable tokens of authority.

The devil is in the details when talking about authentication, authorization and Web APIs. When I first heard about the Yahoo's proposed authentication model for Web APIs at their ETech 2006 talk entitled Building a Participation Platform: Yahoo! Web Services Past, Present, and Future, I thought it sounded similar to the model used by Passport Windows Live ID. In both approaches instead of applications prompting users for their credentials (username/password combo), the user signs in to the primary service which then returns an opaque token to the target application that identifies the user and gives the application permission to access the user's data. However, having a fine grained access that can give applications access only specific services and can revoke permission given to specific applications seems to be richer than what I've seen offered by  Passport Windows Live ID. This is nice but it's to be seen how easy this will be for users to understand or for applications to manage.

From my perspective there are two primary goals an authentication model for a family of Web APIs must satisfy

1. User credentials are sacred and must be protected at all costs: A security mechanism is only as strong as its weakest link. This means that it is extremely unwise to build an authentication model that has applications built on your APIs to request username/passwords or other credentials from users directly. The last thing you want is for anyone with a copy of Javascript for Dummies to be able to legitimately ask your users for their credentials then store them insecurely. In addition, if users get comfortable with entering their credentials in all sorts of random places then it makes them more susceptible to phishing attacks. This is one of the reasons services like Meebo are worrying to me.

   It should be noted that in certain cases, the information hosted in the service may not be very valuable in which case this tennet can be waived. For example, the NewsGator API expects applications to prompt users for their credentials and then pass those along when interacting with the service. Since the user information hosted in the service is primarily a list of RSS/Atom feeds and their read/unread state, the value to attackers is extremely low and there is little need to build a sophisticated authentication model for this service.

2. Do not discriminate against any platform or any device: In todays world, end users interact with online services using a variety of devices and platforms. Each device and platform has different strengths and limitations but is important in its own right. Online services like email or instant messaging have witnessed the rise of multiple access models from desktop applications running on PCs to applications running on mobile devices, from JavaScript code running clientside in a browser to web service calls being made from one server to another. In many cases, the average user may go back and forth between all these access modes within the course of their normal usage of the service. For example, I check my email using Outlook Web Access, Microsoft Outlook and my Audiovox SMT 5600 during the normal course of my work day. 

Thus far I have not seen any Web API authentication model satisfy both goals. Based on my understanding from the ETech talk, the model proposed by Yahoo! fails to meet the second goal above because it is browser based. Before being accused of bias, I'll also point out that from reading the initial documentation for the Windows Live ID service it also fails to satisfy the second goal because Microsoft has only announced SDKs for server-to-server calls and desktop clients [both of which I assume will only target servers or PCs running varieties of Windows]. 

Providing a comprehensive authentication story for a suite of Web APIs is a hard problem.

 

The Windows Live Custom Domains team has a post on their blog entitled Bye bye beta….Custom Domains v1 has launched which states

We’re leaving the “Beta tag” blanket at home.  That’s right…thanks to your beta testing and feedback; we’ve now officially launched Windows Live Custom Domains.  Our colleagues over in Messenger kicked off the Windows Live launch season last week.  Along with the launch of OneCare and Live Favorites, we're excited to continue the momentum.  Windows Live is about the Web, the way you want it.  Personalization is a key piece here, and let's face it...your identity online is central to that.  Custom Domains enables people to use all of the Windows Live and MSN services they want with an ID that's as unique to them as they want it to be.

For those who aren’t familiar with Custom Domains, we provide free hosted e-mail for your domain.  Let’s say you own the domain name, “wingtiptoys.com.”   With Custom Domains, you get unlimited, free e-mail accounts at that domain.  You can open accounts for sales@wingtiptoys.com, owner@wingtiptoys.com, etc.  Oh wait, did we mention that it’s free?  This isn’t one of those “free during beta” trial offers.  This is free for life. 

New Feature: Open Membership

We’re jazzed about a new feature we’ve added called Open Membership.  How does this work?  Let’s say you run a website called “soccerfan.com.”  Your users love your site and want an e-mail address @soccerfan.com.  Prior to this launch, each user would have to request an e-mail account from the administrator.  Then, the admin would manually approve and create each account.  We’ve made things much easier all around with this launch...With the Open Membership featured enabled, we provide URL links so users can automatically sign-up for an e-mail account @soccerfan.com.  Admins no longer have to burden with the manual creation of email accounts, and users get accounts immediately.

Congratulations to the team, it's good to see more Windows Live services coming out of beta. I really like this service but I'd love to see it expand to cover other Windows Live properties. For example, will I be able to ever use my own custom domain for my Windows Live Space?

 

The Windows Live Local/Virtual Earth team has a blog post entitled Free Phone calls at WLL which states

A new release of Live Local went out over the weekend. Mostly minor bug fixes, but a few new features made it in as well. One of the more interesting is the ability to phone any business for free. Using it is easy - do a business search by name or category and in the result panel will be a 'Call for Free' link next to each business listing. Each pushpin popup on the map will also have the Call for free link. When you click it you specify your phone number  -the system will dial both you and the business and connect you. Once you've made your first call, you can rapid dial businesses without having to re-enter your phone number.

Windows Live Local is definitely my favorite online mapping service today and probably the only Windows Live service I can say is head and shoulders above the competition. Kudos to everyone on the team who have built such a killer service in such a short time.

 

Quentin Clark has posted a new entry entitled Update to the Update on the WinFS team blog that answers some of the questions that have been raised since his post on Friday about the status of WinFS. He writes

Is WinFS dead?
Yes and No. Yes, we are not going to ship WinFS as a separate, monolithic software component. But the answer is also No - the vision remains alive and we are moving the technology forward. A lot of the technology really was database stuff – and we’re putting that into SQL and ADO. But some of the technology, especially the end user value points, are not ready, and we’re going to continue to work on that in incubation. Some or all of these technologies may be used by other Microsoft products going forward.

Does your plan for WinFS have any impact on Windows Vista?
There is no impact on Windows Vista. We announced back in August 2004 that WinFS would not be in Windows Vista.

Will the "Relational Filesystem" ever be in Windows?
Hey – we are very busy finishing Vista, and just aren’t ready to talk about what comes next. The vision for a richer storage in Windows is very much alive.  With the new tools for searching and organizing information in Windows Vista, we are taking a good step towards that vision.  

Why are parts of WinFS going into SQL Server?
We have a vision around data that guides us we call the "Data Platform Vision". We’ve been talking with customers about this for some time, and we have heard consistent positive feedback. It was clear that the integrated storage and automation features of WinFS will help SQL Server deliver on the "Beyond Relational" and "Continuous Availability and Automation" promises of that vision. We decided to focus resources on delivering these technologies to our customers as part of the Data Platform Vision in the near term.

Why did Microsoft announce this now after talking about WinFS at TechEd so recently?
When we were at TechEd, we had not made the decision. Sure, it was under discussion, but we did not have all the information we needed and we had not made the call yet. We did share the news as soon as we had the final word. We could have waited longer to disclose the information and made the change in plans less of a contrast, but we chose to notify people as soon as we could. This is why we used the blog and didn’t fire-up the big MS PR machinery – that takes time.

I commented internally that the response to Quentin's original blog post shows that there has been a discrepancy between what the WinFS team has been working on and what the developer community believes they were delivering. I got to read a draft of this blog post before it went up and it does a better job of stating what has happened with WinFS and even seems to have incorporated some of my feedback. I hope Charles Miller doesn't think this post is also un-blog-like. :)

 

I was reading Charles Miller's post entitled We Come to Bury WinFS... where he wrote

The first thing to strike me about the blog post announcing the end of WinFS as a Vista feature is how totally un-blog-like it is.

Every comment (bar one) got the point. WinFS is dead. Its carcass is being split between SQL Server and ADO.NET, and the relational filesystem that was going to change the way we use computers is no longer just postponed to be shipped after Vista, it’s gone.

The blog post itself, however, is written entirely in marketing-speak. The engineer talks about how super-excited the team is about this "new direction", how encouraging this news is, and leaves the fate of Vista for a final, particularly obfuscatory paragraph. Nary a word is allowed to suggest that the last nail in the coffin for Vista’s most eagerly anticipated feature might be a huge let-down to those people who have been watching it slip further and further down the schedule since its fanfare announcement as a part of Longhorn three years ago.

Did Microsoft forget everything Scoble was supposed to be teaching them, so quickly?

Every now and then, you’ve got to put out a mea culpa. You’ve promised something that turned into something else, or that you changed your mind about, or that you just can’t deliver. In the mass-media world, you do this by spinning the story as positively as you can. The message will be filtered by intermediaries before it reaches the public, and it’s expected the journalists in the middle will get the point, pulling quotes from the positive spin to offset the otherwise negative message.

I agree 100% with Charles Miller's sentiments about the blog posting on WinFS. This seems to be another example a case where Microsoft overpromised and underdelivered failed to deliver but even worse instead of owning up to this, the blog post spins this as being what customers want. From reading the hundred or so comments and trackbacks to Quentin Clark's post it doesn't seem that there are many people who are excited or encouraged that what once was touted as a pillar of Longhorn is now just another checkbox feature in SQL Server. This makes Microsoft look bad to developers because it means that we are either insulting our developer customers by thinking we can pull the wool over their eyes in such a blatant way or even worse that we are completely out of touch with them. Either way, it sucks and I feel like we should apologise to developers and perhaps even the software industry as a whole. Microsoft did offer a mea culpa to developers for the delay between Internet Explorer versions and I think this is another one of those cases where we should do the same.

I feel like I should probably throw in some last thoughts about WinFS, the technology, especially since in my previous post I claim that this decision should have been made a few years ago. Below is a random collection of my thoughts WinFS which can also be gleaned from my various blog posts on the technology over the last few years

1. There was a divergence of opinion in what the team was building and what the people thought the team was building. A common misconception was that WinFS would somehow make search "better" on the desktop in the same way that desktop search tools like Lookout do. I addressed this misconception further in my post Killing the "WinFS is About Making Search Better" Myth from almost two years ago.

2. The project had the biggest example of scope creep I'd ever seen at Microsoft. When WinFS swallowed ObjectSpaces the team decided that instead of just tackling the hard problem of building an object/relational file system for Windows that they also wanted to tackle the hard problem of  building an enterprise-class object to relational mapping technology with the same product in a single release. It also didn't help that key ObjectSpaces folks like Matt Warren, Dinesh Kulkarni and Luca Bolognese ended up joining the C# team to work on LINQ which meant WinFS inherited all of the problems of ObjectSpaces but not necessarily all the folks who had been working on those problems.
   

3. The chicken and the egg problem. One of the key ideas in the WinFS type system for the Windows desktop was that we'd have common file system level types for high level concepts like people/contacts, email messages or photos instead of just opaque bits on disk with some file format specific metadata. To take advantage of this, existing applications would have to be rewritten or new [backwards incompatible] applications would have to be written targetting WinFS. The primary benefits of making this change [besides the improved programming model] were the network effects if lots of applications used these types (e.g. Outlook stored mail identified by a WinFS contact, RSS Bandit stored RSS feeds from that WinFS contact, AOL Instant Messenger stored IM conversation logs using that contact, etc). Even if you got these network effects you then had to deal with the Windows registry problem (i.e. apps stomping over each others data which is one of the main problems with the Windows registry today).
   

4. I never saw good answers to the questions Jon Udell asked in his blog posts Questions about Longhorn, part 1: WinFS and Questions about Longhorn, part 2: WinFS and semantics. Specifically, the world is betting big on open file formats such as XML including parts of Microsoft (e.g. Microsoft Office) so why would anyone want to build aplications targetting a proprietary Windows-only file system that didn't have a good XML story for getting data out of the platform?

I should probably stop now. Even though all the information above is freely available to anyone who reads blogs and can put two and two together, some may object to the above collection of thoughts.
 

Quentin Clark has a blog post entitled WinFS Update where he writes

There are many great technical innovations the WinFS project has created – innovations that go beyond just the WinFS vision but are part of a broader Data Platform Vision the company is pursuing.  The most visible example of this today is the work we are now doing in the next version of ADO.NET for Orcas.  The Entities features we are now building in ADO.NET started as things we were building for the WinFS API.  We got far enough along and were pushed on the general applicability of the work that we made the choice to not have it be just about WinFS but make it more general purpose (as an aside – this stuff is really coming together – super cool). 

Other technical work in the WinFS project is at a similar point – specifically the integration of unstructured data into the relational database, and automation innovations that make the database "just work" with no DBAs – "richer store" work.  It's these storage innovations that have matured to the point where we are ready to start working on including them in our broader database product.  We are choosing now to take the unstructured data support and auto-admin work and deliver it in the next release of MS SQL Server, codenamed Katmai.  This really is a big deal – productizing these innovations into the mainline data products makes a big contribution toward the Data Platform Vision we have been talking about.  Doing this also gives us the right data platform for further innovations. 

These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release.  With most of our effort now working towards productizing mature aspects of the WinFS project into SQL and ADO.NET, we do not need to deliver a separate WinFS offering. 

So that's it, no more WinFS. This is the right decision, albeit two years too late but better late than never. It's sad to think about the projects that got killed or disrupted because of WinFS only for this to happen. In a recent column entitled Taking One for the Team Robert X. Cringley has a quote from Management By Baseball by Jeff Angus which reads "When I worked for a few years at Microsoft Corporation in the early '80s,...no one cared to track and codify past failures as a way to help managers create guidelines of paths to follow and avoid". I hope this doesn't end up happening with the lessons from the WinFS project.

 

Mike Champion has a blog post entitled Why does the world need another XML API? where he writes

One basic question keeps coming up, something like: "We have SAX, DOM, XmlReader/Writer APIs (and the Java people have a bunch more), we have XSLT, we have XQuery ... why do you think we need Yet Another XML API?"
...

• XmlReader / XmlWriter can't go away because XLinq uses them to parse and serialize between XLinq objects and XML text. Also, while we are making XLinq as streaming-friendly as possible (see the XStreamingElement class in the CTP release for a taste of where we are going), we're only aiming at hitting the 80/20 point...
• DOM can't go away because there are important use cases for API-level interoperability, most notably in the browser...DOM doesn't make code truly interoperable across implementations (especially on other languages), but there is enough conceptual similarity that porting is generally not terribly difficult...  
• XSLT definitely won't go away. The Microsoft XML team was promoting XQuery as a "better XSLT than XSLT 2.0" a few years ago (before I came, don't hurt me!), and got set straight by the large and vocal XSLT user community on why this is not going to fly. While it may be true in some abstract way that XQuery or XLinq might logically be able to do everything that XSLT does, as a practical matter it won't...  
  
• XQuery won't go away, at least for its original use case as a database query language.  Microsoft supports a draft of XQuery in SQL Server 2005, contributes to the work of the XQuery working group at W3C, and will continue to invest in finalizing the XQuery Recommendation and implementing it in our DBMS..
  

we believe that the overall LINQ story is going to have a pretty profound impact on data programmability, and we want to make sure that LINQ has a good story for XML...For XML users, I see a few really powerful implications:

• The ability to query data by declaraing the characterics of the result set rather than imperatively navigating through and filtering out all the data...
• The ability to join across diverse data sources, be they XML documents, objects, or DBMS queries
• The ability to "functionally" reshape data within the same language as the application is written.  XSLT pioneered the functional transformation approach to XML processing, but it is difficult for many developers to learn and requires a processing pipeline architecture to combine XSLT transforms with conventional application logic...

This brings back memories of my days on the XML team at Microsoft. We went back and forth a lot about building the "perfect XML API", the one problem we had was that there one too many diverse user bases which had different ideas of what was important to expose in an API. We were always caught between a rock and a hard place when it came to customer requests for fixing our APIs. To some people (e.g. Microsoft Office) XML was a markup format for documents while to others (e.g. Windows Communications Foundation aka Indigo) it was simply a serialization format for programming language objects. Some of our customers were primarily interested in processing XML in a streaming fashion (e.g. Biztalk) while others (e.g. Internet Explorer) always worked on in-memory XML documents. Then there were the teams whose primarily interest was in strongly typed XML (e.g. SQL Server, ADO.NET) since it would be stored in relational database columns.

In trying to solve all of these problems with a single set of APIs, we went down the road of prematurely declaring the death of certain XML APIs and technologies such as the DOM (see Ode to the XML DOM) and XSLT (see XSLT 2.0 Sir? or Why You Won't See XSLT 2.0 or XPath 2.0 in the Next Version of the .NET Framework). At the end of the day we saw the light and we eventually changed our tune by not deprecating the System.Xml.XmlDocument class and by reconsidering whether replacing XSLT with XQuery was the right way forward.

When I was on the team there was a lot of infatuation with XQuery which eventually turned to frustration. There were a number of technical and non-technical reasons for this such as its dependence on W3C XML Schema which significantly complicated its type system and how long the spec was taking to become a standard (over 5 years and counting as I write this post). Since then a bunch of folks who were were enamored with XQuery have taken some of its lessons (e.g. declaritiveness, simple model for XML generation, etc) and integrated it into a mainstream programming environment with the XLinq project. XML geeks everywhere should read Erik Meijer's paper, XLinq: XML Programming Refactored (The Return Of The Monoids), it is a thing of beauty if angle brackets are your thing. And even better, if you are one you are one of those that chants rabid slogans like "XML is the assembly language of Web 2.0", you'll still like XLinq because it provides a easier and richer level of abstraction for working with XML.

Enjoy.

 

One of my coworkers sent me a link to the blog post PhotoBucket Leads Photo Sharing Sites; Flickr at #6 on the HitWise company blog. The highlights of the post are excerpted below

In the SF tech bubble that I live in, most of the talk about photo sites has been centered on Flickr. In fact, you could get the impression from most people I meet that Flickr is the ONLY site at which you can share and store photos. Examination of the category however, shows that Flickr is #6 among the top 10 photo sharing sites, with a market share of 5.95%. Industry standbys like Yahoo! Photos, Webshots Community, and Kodak Gallery currently rank higher than Flickr.

Photobucket dominates the category, with a 44% market share. It surpassed Yahoo! Photos in January, and its share of visits increased by 34% in the four months from February 2006 to May 2006. Flickr, my friends should be happy to note, has also been growing rapidly, increasing 44% in the past four months, and up from a rank of #9 in this category one year ago (week ending 6/18/05). Slide has also taken off this spring, with its visits increasing more than ten fold in the past four months.

In the comments, someone asked about the methodology and why the HitWise marketshare numbers differ significantly from those of other ratings companies like ComScore. The response is that the HitWise numbers are based on page views from sampling 10 million users while ComScore numbers are for unique users. According the the comment ComScore states that both Flickr and Photobucket  get about 16.5 million unique visitors a month while Yahoo! Photos gets about twice that number. Besides the page views versus unique users distinction, another thing that makes this somewhat of an apples to oranges comparison is that the HitWise sample is for Internet users in the USA while ComScore is talking about worldwide usage.

However this is still an instructive set of statistics. For one it shows that even if Flickr does have more people talking about it in the tech blogosphere, Photobucket is generating a lot more page views [mainly through MySpace integration]. Secondly, it shows the value of an integrated suite of social applications [photo sharing, social networking, video sharing, blogging, etc] in engaging users. Flickr may or may not have just as many visitors as Photobucket but it's clear that the average visitor to Photobucket views significantly more photos from the site than the average Flickr visitor. This is primarily due to the fact that Photobucket is one of the top image hosting sites used by MySpace users. This seems to validate the approach of building an integrated social software application like MSN Spaces or Yahoo! 360 instead of a mishmash of narrowly tailored social software applications. Where I think Microsoft and Yahoo! have gone wrong and MySpace has gotten it right is that they rely a lot on an ecosystem of supporting sites for extra features (e.g. image hosting, video hosting, etc) instead of trying to do it all in-house. This enables them to innovate a lot faster since Microsoft and Yahoo! are then competing with multiple companies instead of just one.

 

Tim Berners-Lee has a blog post entitled Net Neutrality: This is serious where he writes

When I invented the Web, I didn't have to ask anyone's permission. Now, hundreds of millions of people are using it freely. I am worried that that is going end in the USA.

I blogged on net neutrality before, and so did a lot of other people. (see e.g. Danny Weitzner, SaveTheInternet.com, etc.) Since then, some telecommunications companies spent a lot of money on public relations and TV ads, and the US House seems to have wavered from the path of preserving net neutrality. There has been some misinformation spread about. So here are some clarifications. ( real video Mpegs to come)

Net neutrality is this:

If I pay to connect to the Net with a certain quality of service, and you pay to connect with that or greater quality of service, then we can communicate at that level.

That's all. Its up to the ISPs to make sure they interoperate so that that happens.

Net Neutrality is NOT asking for the internet for free.

Net Neutrality is NOT saying that one shouldn't pay more money for high quality of service. We always have, and we always will.

I've been depressed by the what has been happening in Washington with regards to Net Neutrality but have hesitated to blog about it since I am clearly biased, I work for a company that is likely to financially benefit if Net Neutrality is the status quo. However that doesn't change the fact that what the telcos are striving would eventually turn the Web into a very different place from what it has been. Read all of Tim Berners-Lee's post and after that  you should read what Lawrence Lessig had to say about this topic in his article for the Washington Post; No Tolls on the Internet.

 

I've counted myself lucky to find a girlfriend who has similar tastes in music to me. Most of my friends don't like equal parts Lil Jon, Metallica, G-Unit and Backstreet Boys but my girlfriend does. In addition, she likes the crunk hip hop clubs just like I do. In fact, the clubs we tend to like going to regularly seem to end up getting closed down for being too crunk (e.g. Mr Lucky, Larry's).

So you can imagine that I was quite surprised by her recent request that we attend a Kenny Chesney concert. So I asked here "Who's Kenny Chesney?" and she replied that he was a country singer. I was like "But you don't like country music" and she replied that actually she did. Since this revelation, you are now just as likely to hear us listening to Settle for a Slowdown or What Hurts the Most as you are to hear Poppin' My Collar or Hustlin' as we drive down the street. How times change.

The concert is this weekend and I'm not sure I have appropriate attire. If one of my coworkers who reads my blog has a cowboy hat that I can borrow, I'd greatly appreciate it.