At Chris Sells' XML DevCon conference* Don Box gave a talk called WS-Why which is described below

"Why? This talk will make sense of why various WS-* specs came to life and which ones every developer should ignore. Naturally, the size of this set is non-zero, however, it is not the entire universe. Hopefully, the audience will be left with a mental model for what to ignore going forward as the WS-* machine continues to move forward."

I got to hang out with Don before the conference as well as read the slides for his talk and although I liked the direction of the talk I wished it could have been more prescriptive. Before continuing to read it's a good idea to read a summary of Don's talk such as the one  at Jeff Barr's blog post AXDC - Don Box and WS-Why?.

In his talk Don breaks XML Web Services specs into

  • WS-DesertIsland - specs that are a must have that form the core XML Web Service specs. These include XML, SOAP, WS-Addressing, WS-MetadataExchange &  XSD+WSDL
  • WS-IslandResort - the next layer of important specs after the core. These include WS-Security, WS-Trust, WS-ReliableMessaging & WS-Policy
  • WS-NewZealand - specs you'd probably need once in a lifetime. These include WS-Eventing, WS-Enumeration & WS-AtomicTransaction
  • WS-IslandOfDoctorMoreau - the ugly step children of the WS-* spec family. These include UDDI, WS-Transfer, WS-BusinessActivity, MTOM and BPEL4WS
  • WS-FantasyIsland - specs Don would love to see. These include WS-Data (XPath/SQL-like query for web services), SOAP over TCP, XSD with better support for versioning, binary XML & WSDL based on RELAX NG.

As can be expected there have been folks who've done a deeper analysis of Don's talk than what I've done above. The most significant so far has been Steve Maine's post The Web Services Kernel which gives an overview of the 5 specs in Don's WS-DesertIsland.

In general I agree with the direction Don took with the talk. However although it was an appropriate talk for the audience of the XML Devcon, a bunch of implementers and industry experts, I don't see it as significant guidance to developers trying to make sense of the mess of WS-* specs. Don's talk is best seen through the lens of "If I was an implementer which specs should I implement in my XML Web Services toolkit", not "If I was a developer which specs should I use from my XML Web Services toolkit". This is an important distinction. This is why specs like WS-Addressing are in Don's WS kernel even though it is only important if you aren't using HTTP as your transport which most developers would be. 

The talk I'd most love to see next from Don or whoever else in Indigo is going to be doing the conference route next is WS-Which: How to decide what XML Web Services specs are right for your application. As someone who now has the responsibility of designing XML Web Service end points within an intranet (aren't job switches grand?) and perhaps beyond I'm interested in guidance that explains when I should use WS-Security versus SOAP + HTTPS or whether there are any scenarios where using MTOM or WS-Transfer actually make sense.

Don's talk was a good start by the Indigo team for providing guidance for future users of the XML Web Services family of specs but there's still a lot of guidance that is currently missing from them to the industry. More importantly, just because some spec is or isn't in Don's WS kernel doesn't indicate how significant or not it will be to a particular class of application developers. Perhaps I can get Doug to give this talk next year.

* Someone really needs to explain to Chris Sells how the Web works. Constantly changing the content of the page at http://www.sellsbrothers.com/conference logically breaks links to the site


 

Categories: XML

October 24, 2004
@ 09:45 PM

Friday was my last day on the XML team at Microsoft. On Monday I start work as a Program Manager on MSN Communication Services Platform team. This team is responsible for the server side implementation of several aspects of MSN including MSN Messenger, some of Hotmail and MSN Groups.

My immediate responsibilities will involve working on the back end of MSN Spaces which is a hosted blogging service in the same vein as Blogger, LiveJournal and TypePad. The site is currently in beta for the Japanese audience.

This means I'll no longer be responsible for the XML Developer Center on MSDN which now will be managed by Irwin Dolobowsky. My work-related blog will shortly no longer appear on that page. However I will continue to write the Extreme XML column on a bi-monthly basis. Arpan Desai is the contact person for the aspects of the XML programming model in the .NET Framework I was responsible for, specifically the System.Xml.XPath namespace, System.Xml.Schema namespace and the System.Xml.XmlNode hierarchy.

If you are curious as to why I decided to make this move, some of the thinking behind my decision can be gleaned from my weblog post Social Software is the Platform of the Future

Update: This move does not change my status as the project lead on RSS Bandit.


 

Categories: Life in the B0rg Cube

October 24, 2004
@ 02:53 AM

A couple of people at work use the term strategy tax when referring to compromises or similar decisions a product team makes that are less than optimal because they have to satisfy the corporate strategy. More and more, I see this term coming up with regards to what Microsoft has termed "integrated innovation". If you are unfamiliar with this term a good description of it can be found in the Microsoft press release 'Integrated Innovation' Provides Partners with Roadmap to Success  where its states

The pursuit of Integrated Innovaton pertains both to product and process. Integrated innovation is manifested in the Microsoft platform but it extends beyond the technology stack. It's Microsoft's approach to building software: simultaneously innovating in product design and development and tightly integrating products, each one with all the others.

Before pointing out the obvious cons of such an approach its benefits should be highlighted. Many would argue that the reason Microsoft Office became the multi-billion dollar business it is today is because it applied to doctrine of integrated innovation. Microsoft took what used to be considered disparate applications (word processing, presentation software, spreadsheets, etc) and turned them into a cohesive productivity suite that worked seamlessly together. The fact that one can design a chart in Excel then cut and paste it into Word with no problems was a big deal when it was first implemented and it still is today. As the saying goes, a lot of effort went into making this look effortless. This combined with the fact that you could purchase these tools in a single box made it the killer application for businesses all over the world.

With that out of the way, let's discuss the cons of this approach. Many students of software engineering will tell you that the goal of any systems architect is to increase cohesion and reduce the coupling in the software architecture. The problems with tight coupling (or tight integration) are well known. As mentioned in Object Oriented Software Engineering Knowledge Base, tight coupling causes a system to be hard to understand and change because changes in one place will require changes somewhere else. Requiring changes to be made in more than one place is problematic since it is time-consuming to find the different places that need changing, and it is likely that errors will be made. 

Those are developer-centric reasons for why tight integration is disadvantegeous but what do these mean in practical terms? The first thing it means is projects take longer. This is basically the Mythical Man Month effect. If you have two teams of 10 people working on separate projects that will take six months to complete and tell them to combine their efforts, it will take them more time to complete their joint tasks not less. As implied above this also means a bug in one component is a bug in all the components, a security flaw in one component is a security flaw in all the components and a slip in one component's schedule is a slip in the schedule for all the components. Finally, it makes the systems more complex to manage and understand which means it makes them harder to maintain.

In today's world where the majority of Microsoft's software is competing with the products of the Open Source community whose mantra of "Release early, release often" and increased threats from malicious hackers make security a primary concern the doctrine of integrated innovation seems almost anachronistic.

The effects of integrated innovation on product schedules shouldn't be underestimated. Although Microsoft Office is an example of integrated innovation when considering its constituent parts it also is an example of eschewing this practice when taken as a whole. The Office team releases a cohesive product with minimal dependencies on external products. The Office team has also consistently released software every 2 to 3 years over the past decade; Office 95, Office 97, Office 2000, Office XP, and Office 2003. Certain other products at Microsoft which are at the forefront of integrated innovation do not seem to have such consistent release cycles. These products are paying the strategy tax.

The main reason for writing this blog entry is so that next time someone asks me what I mean by "strategy tax" I can just give them a link to this entry instead of repeating myself.


 

Categories: Life in the B0rg Cube

October 22, 2004
@ 05:01 AM

It's hard to tell the headlines Google misses earnings expectations and Google's 3Q Profit More Than Doubles  are about the same occurence. I guess it's all about perspective. The market seems to be taking the half full approach seeing as GOOG jumped by $8.89 to close at $149.38.

Even more impressive is that it is trading at $161.79 in after hours trading. And I thought the original IPO price of $135 was excessive. Dang.


 

Karl Waclaweck has released version 1.0 of the SAX for .NET project. In the announcement on XML-DEV Karl writes

This is the first production release of the C#/.NET port of the SAX API.
It should be compatible with MS.NET 1.1 and Mono 1.0.2.

Since the API alone is not enough, a SAX parser implementation has been
released as well: SAXExpat.NET 1.0. It is based on the Expat parser, and
will work on MS.NET 1.1. Currently Mono 1.0.2 is not able to run it,
but this will hopefully change with future Mono releases.

Another implementation based on a port of the AElfred parser will
be available soon. It should work under both, MS.NET and Mono.

The project page is here: http://saxdotnet.sourceforge.net/

It's good to see more Open Source XML projects showing up for the .NET Framework. I haven't missed using SAX that much but I imagine people coming to the .NET Framework from the Java world would love to be able to keep using their favorite push model XML parsing API.


 

Categories: XML

It seems about half the feeds in my aggregator are buzzing with news of the new Google Desktop Search. Although I don't really have the need for a desktop search product I was going to download it and try it out anyway until I found it user a web browser interface accessed via a local web server. Not being a fan of browser based user interfaces I decided to pass. Since then I've seen a couple of posts from people like Joe Gregorio and Julia Lerman who've claimed that Google Desktop Search delivers the promise of WinFS today.

Full text search is really orthogonal to what WinFS is supposed to enable on the Windows platform. I've written about such misconceptions in the past, most recently in my post Killing the "WinFS is About Making Search Better" Myth where I wrote

At its core, WinFS was about storing strongly typed objects in the file system instead of opaque blobs of bits. The purpose of doing this was to make accessing and manipulating the content and metadata of these files simpler and more consistent. For example, instead of having to know how to manipulate JPEG, TIFF, GIF and BMP files there would just be a Photo item type that applications would have to deal with. Similarly one could imagine just interacting with a built in Music item instead of programming against MP3, WMA, OGG, AAC, and WAV files. In talking to Mike Deem a few months ago and recently seeing Bill Gates discuss his vision for WinFS to folks in our building a few weeks ago it is clear to me that the major benefits of WinFS to end users is the possibilities it creates in user interfaces for data organization.

Recently I switched from using WinAmp to iTunes on the strength of the music organizational capabilities of the iTunes library and "smart playlists". The strength of iTunes is that it provides a consistent interface to interacting with music files regardless of their underlying type (AAC, MP3, etc) and provides ways to add metadata about these music files (ratings, number of times played) then organize these files according to this metadata. Another application that shows the power of the data organization based on rich, structured metadata is Search Folders in Outlook 2003. When I used to think of WinFS I got excited about being able to perform SQL-like queries over items in the file system. Then I heard Bill Gates and Mike Deem speak about WinFS then saw them getting excited about the ability to take the data organizational capabilities of features like the My Pictures and My Music folders in Windows to the next level it all clicked.

Now this isn't to say that there aren't some searches made better by coming up with a consistent way to interact with certain file types and providing structured metadata about these files. For example a search like

Get me all the songs [regardless of file type] either featuring or created by G-Unit or any of its members (Young Buck, 50 Cent, Tony Yayo or Lloyd Banks) between 2002 and 2004 on my hard drive

is made possible with this system. However it is more likely that I want to navigate this in a UI like the iTunes media library than I want to type the equivalent of SQL queries over my file system.

Technologies like Google Desktop Search solve a problem a few people have while WinFS is aimed at solving a problem most computer users have. The problem the Google Desktop Search mainly satisfies is how to locate a single file in your file system that may not be easy to navigate to via the traditional file system explorer. However most computer users put files in few locations on their file system so they usually know where to find the file they need. Most of the time I put files in one of four folders on my hard drive 

  1. My Documents
  2. My Music
  3. Visual Studio Projects (subfolder of My Documents)
  4. My Download Files 

For some of my friends you can substitute the "Visual Studio Projects" folder for the "My Pictures" folder. I also know a number of people who just drop everything on their Windows desktop. However the point is still the same, lots of computer users store a large amount of their content in a single location where it eventually becomes hard to manipulate, organize and visualize the hundreds of files contained therein. The main reason I stopped using WinAmp was that the data organization features of Windows Explorer are so poor. Basically all I have when dealing with music files is 'sort by type' or some variation of 'sort by name' and a list view. iTunes changed the way I listened to music because it made it extremely easy to visualize and navigate my music library. The ability to also perform rich ad-hoc queries via Smart Playlists is also powerful but a feature I rarely use.

Tools like Lookout and Google Desktop Search are a crutch to get around the fact that the file navigation metaphor on most desktop systems is past its prime and is in dire need of improvement. This isn't to say fast full text search isn't important, even with all the data organizational capabilities of Microsoft Outlook I still tend to use Lookout when looking for emails sent past a few weeks ago. However it is not the high order bit in solving the problems most computer users have with locating and interacting with the files on their hard drives.

The promise of WinFS is that it aims to turn every application [including file navigation applications like Windows explorer] into the equivalent of Outlook and iTunes when it comes to data visualization and navigation by baking such functionality into the file system APIs and data model. Trying to reduce that to "full text search plus indexing" is missing the forest for the trees. Sure that may get you part of the way but in the end it's like driving a car with your feet. There is a better way and it is much closer than most people think.


 

Categories: Technology

In his post The gender profile of Wikipedia Joi ito writes

I haven't conducted any scientific analysis or anything, but Wikipedia seems much more gender balanced than the blogging community. I know many people point out that ratio of men at conferences on blogging and ratio of men who have loud blog voices seems to be quite high.

The core mistake in the assumption Joi Ito makes here is in assuming that participation is equivalent to talking about participation. I've seen several statistics and surveys on blogosphere (God, I hate that word) participation which all seem to point to the same conclusion; the number of female bloggers tends to outnumber the number of male bloggers.

For example, according to the LiveJournal statistics page there twice as many females blogging as males in that community. Given that LiveJournal is one of the oldest and largest blogging communities with almost 2 million active blogs (and almost 5 million user accounts) I think this counts for a lot more than claiming that a lot of women aren't seen at geeky conferences like Web 2.0 or Tim O'Reilly's FooCamp.   

One thing I've found interesting is wondering why if there are more female bloggers than male bloggers, lists such as the Technorati Top 100 are dominated by male bloggers. On Friday I attended a talk by Susan Herring which shed some light on the issue entitled Conversations in the blogosphere: An analysis "from the bottom up." where she discussed some research she and her colleagues had undertaken to discover the nature of conversations in the blogosphere. Interesting data points from her presentation include

  • Blogs with more outgoing links tend to be more linked to
  • Women tend to link less than men (even in women-centric blog circles such as homeschooling the rate of linking is comparatively less than male-centric blog circles like warblogging)
  • Linking doesn't tend to be reciprocal
  • Small percentage of blogosphere is interlinked.
  • 67% - 75% of blogs surveyed don't link outwards to other blogs
  • 95% of blogs surveyed have less than 10 inbound links from other blogs

Some of the data points from Susan's presentation gave me some ideas as to why the various lists of popular weblogs always seem to be male dominated.

The methodology and results of Susan's research can be obtained from her paper, Conversations in the blogosphere: An analysis "from the bottom up."


 

Categories: Ramblings

Derek Denny-Brown, the dev lead for both MSXML & System.Xml, who's been involved with XML before it even had a name has finally started a blog. Derek's first XML-related post is Where XML goes astray... which points out three features of XML that turn out to have caused significant problems for users and implementers of XML technologies. He writes

First, some background: XML was originally designed as an evolution of SGML, a simplification that mostly matched a lot of then existing common usage patterns. Most of its creators saw XML and evolving and expanding the role of SGML, namely text markup. XML was primarily intended to support taking a stream of text intended to be interpreted as a human readable document, and delineate portions according to some role. This sequence of characters is a paragraph. That sequence should be displayed with a link to some other information. Et cetera, et cetera. Much of the process in defining XML based on the assumption that the text in an XML document would eventually be exposed for human consumption. You can see this in the rules for what characters are allowed in XML content, what are valid characters in Names, and even in "</tagname>" being required rather than just "</>".
...
Allowed Characters
The logic went something like this: XML is all about marking up text documents, so the characters in an XML document should conform to what Unicode says are reasonable for a text document. That rules out most control characters, and means that surrogate pairs should be checked. All sounds good until you see some of the consequences. For example, most databases allow any character in a text column. What happens when you publish your database as XML? What do you do about values that include characters which are control characters that the XML specification disallowed? XML did not provide any escaping mechanism, and if you ask many XML experts they will tell you to base64 encode your data if it may include invalid characters. It gets worse.

The characters allowed in an XML name are far more limited. Basically, when designing XML, they allowed everything that Unicode (as defined then) considered a ‘letter’ or a ‘number’. Only 2 problems with that: (1) It turns out many characters common in Asian texts were left out of that category by the then-current Unicode specification. (2) The list of characters is sparse and random, making implementation slow and error prone.
...
Whitespace
When we were first coding up MSXML, whitespace was one of our perpetual nightmares. In hand-authored XML documents (the most common form of documents back then), there tended to be a great deal of whitespace. Humans have a hard time reading XML if everything is jammed on one line. We like a tag per line and indenting. All those extra characters, just there so that our feeble minds could make sense of this awkward jumble of characters, ended up contributing significantly to our memory footprint, and caused many problems to our users. Consider this example:
 <customer>  
           <name>Joe Schmoe</name>  
           <addr>123 Seattle Ave</addr> 
  </customer>
A customer coming to XML from a database back ground would normally expect that the first child of the <customer> element would be the <name> element. I can’t explain how many times I had to explain that it was actually a text node with the value newline+tab.
...
XML Namespaces
Namespaces is still, years after its release, a source of problems and disagreement. The XML Namespaces specification is simple and gets the job done with minimum fuss. The problem? It pushes an immense burden of complexity onto the APIs and XML reader/writer implementations. Supporting XML Namespaces introduces significant complexity in the parsers, because it forces parsers to parse the entire start-tag before returning any text information. It complicates XML stores, such as DOM implementations, because the XML Namespace specification only discusses parsing XML, and introduces a number of serious complications to edit scenarios. It complicates XML writers, because it introduces new constraints and ambiguities.

Then there is the issue of the 'default namespace’. I still see regular emails from people confused about why their XPath doesn’t work because of namespace issues. Namespaces is possibly the single largest obstacle for people new to XML.

My experiences as the program manager for the majority of the XML programming model in the .NET Framework agree with this list. The above list hits the 3 most common areas people seem to have problems with working with XML in the .NET Framework. His blog post makes a nice companion piece to my The XML Litmus Test: Understanding When and Why to Use XML article on MSDN.


 

Categories: XML

As has been pointed out by others you can read your GMail inbox in any Atom-enabled aggregator that supports secure feeds. I just subscribed to my GMail inbox in RSS Bandit and it worked like a charm. Screenshot below.


 

Categories: RSS Bandit

We are in the process of locking down System.Xml for Beta 2 of the .NET Framework 2.0 and Visual Studio 2005. In the past few months we have received customer feedback about our feature set previewed in the Whidbey Alpha & Whidbey Beta 1 and this has guided our decision making process as to where to focus our energies to ensure a comprehensive feature set.

Below is the list of changes to System.Xml and subsidiary namespaces that have occurred between Beta 1 and Beta 2 of the .NET Framework 2.0 release.

ADDITIONS

XmlSchemaValidator

The XmlSchemaValidator class provides a push model API for W3C XML Schema validation. The primary scenario for using the XmlSchemaValidator is for validating an XML infoset in-place without having to serialize it as an XML document then reparse the document using a validating XML reader.

CHANGES

XmlReader

  • Added overloads to the static Create() method that take XmlParserContext
  • ReadValueAsXXX() methods renamed to ReadContentAsXXX(). Also reduced the number of ReadContentAsXXX() methods relative to the number of ReadValueAsXXX() methods in Whidbey beta 1.
  • Added ReadElementContentAsXXX() methods which are specific to obtaining the value of element nodes
  • Added methods for reading large streams of text or binary data embedded in an XML document in a streaming fashion

public virtual bool CanReadValueChunk { get; }

public virtual int ReadValueChunk (byte[] buffer, int startIndex, int count);

public virtual bool CanReadBinaryContent { get; }

public virtual int ReadContentAsBase64 (byte[] buffer, int startIndex, int count);

public virtual int ReadContentAsBinHex (byte[] buffer, int startIndex, int count);

public virtual int ReadElementContentAsBase64(byte[] buffer, int startIndex, int count);

public virtual int ReadElementContentAsBinHex(byte[] buffer, int startIndex, int count);

  • Added ReadToFollowing(string localname, string namespaceURI) which moves to the next occurrence of the named element in document order.

XmlReaderSettings

  • Added XmlSchemaValidationFlags enumeration to replace the following flags; IgnoreInlineSchema, IgnoreSchemaLocation, IgnoreValidationWarnings and IgnoreIdentityConstraints
  • Added existing ValidationType enumeration to replace to replace the following flags; DtdValidate and XsdValidate

XmlWriter

  • Reduced number of WriteValue() methods
  • Removed overloads of WriteStartElement and WriteStartAttribute that took an IXmlSchemaInfo parameter

XPathDocument

XPathNavigator and XPathEditableNavigator

  • The XPathEditableNavigator has been merged into the XPathNavigator, making it an editable XML cursor model API.
  • The XPathNavigator is the preferred API for exposing data as XML. This has been incorporated into the design guidelines for using XML in the .NET Framework

XmlDocument

  • The XPathNavigator returned by the CreateNavigator() method now allows one to edit the XmlDocument through the cursor model API.
  • The XmlDocument now supports XML schema validation of the entire subtree or partial validation of nodes in the document using the Validate() method
  • The following property added to XmlDocument

public XmlSchemaSet Schemas { get; set; }

  • The following property added to XmlNode

public virtual IXmlSchemaInfo SchemaInfo { get; }

XsltCommand

  • The XslTransform class was obsoleted in Whidbey Beta 1 and replaced by the System.Xml.Query.XsltCommand class.  In Beta 2, we decided to revamp the XsltCommand API in order to make migration from XslTransform simpler.  This effort also resulted in the renaming of the XsltCommand class to  System.Xml.Xsl.XslCompiledTransform.
  • XslCompiledTransform compiles XSLT to MSIL for significantly improved performance at the cost of increased (yet still small) compile times.
  • Supports the MSXML XSLT extension functions such as format-date, format-time etc.

Inference

  • This class has been renamed to XmlSchemaInference

XPathExpression -

  • Added static Compile() method enables one to compile an input string containing an XPath query into an XPathExpression object

REMOVALS

XmlArgumentList

To reduce the cost of churn caused by the obsoletion of XslTransform this class has been removed. In its place the XsltArgumentList from v1.1 can be used

XQueryCommand

Microsoft has decided not to ship a client side XQuery implementation in .NET Framework 2.0 as our customers expect us to ship an implementation that meets the following criteria:

  • Compliant with the W3C standards
  • Functionally addresses key scenarios

As a core platform component in Windows, they also expect us to ship a product that meets the high bar of not breaking their applications when future updates are released.  After talking to key customers and partners, we have determined it is important that we cross this high bar before shipping a full implementation of XQuery in the platform. 

The best estimates tell us that ETA for XQuery to become a W3C recommendation is end of 2005 which does not fit with the .NET Framework 2.0 product release cycle.

In the meantime, we are shipping a well-defined small subset of XQuery in SQL Server 2005 to query information stored natively as XML data type.  This will enable new customer scenarios in SQL Server for storing and retrieving semi-structured data.

In the NET Framework 2.0 RTM timeframe, we recommend that our customers continue to use XSLT and XPath on the client side to solve their key client side filtering and transformation scenarios.  With this in mind, we have made significant improvements to our client side story including:

  • Performance improvements - making the .NET Framework XSLT processor the best performing processor.
  • Functional improvements - improving the usability and feature set of the existing .NET Framework processor

Note: As a result of not shipping XQuery, XML Views using mapping and XQuery to query SQL Server 2005 and the XmlAdapter to perform updates that were originally previewed in the PDC Alpha release of .NET V2.0 have also been removed. These were removed in the Beta 1 release.


 

Categories: Life in the B0rg Cube | XML