These are my notes from the Odeo -- Podcasting for Everyone session by Evan Williams.

Evan Williams was the founder of Blogger and Odeo is his new venture. Just as in his post How Odeo Happened Evan likens podcasting to audioblogging and jokingly states that he and Noah Glass invented podcasting with AudioBlogger. Of course, the audience knew he was joking and laughed accordingly. I do wonder though, how many people think that podcasting is simply audioblogging instead of realizing that the true innovation is the time shifting of digital media to the user's player of choice.

The Odeo interface has three buttons across the top; Listen, Sync and Create. Users can choose to listen to a podcast from a directory of podcasts on the site directly from the Web page. They can choose to sync podcasts from the directory down to their iPod using a Web download tool which also creates Odeo specific playlists in iTunes. 

The Odeo directory also contains podcasts that were not created on the site so they can be streamed to users. If third parties would rather not have their podcasts hosted on Odeo they can ask for them to be taken down.  

The Create feature was most interesting. The website allows users to record audio directly on the website without needing any desktop software. This functionality seems to be built with Flash. Users can also save audio or upload MP3s from their hard drive which can then be spliced into their audio recordings. However one cannot mix multiple audio tracks at once (i.e. I can't create an audio post then add in background music later, I can only append new audio).

The revenue model for the site will most likely be by providing hosting and creating services that allow people to charge for access to their podcasts. There was some discussion on hosting music but Evan pointed out that there were already several music sites on the Web andd they didn't want to be yet another one.

Odeo will likely be launching in a few weeks but will be invitation-only at first.


 

This was a late breaking session that was announced shortly after The Long Tail: Conversation with Chris Anderson and Joe Kraus. Unfortunately, I didn't take my tablet PC with me to the long tail session so I don't have any notes from it. Anyway, back to Google Code.

The Google Code session was hosted by Chris DiBona. The Google Code homepage is is similar to YSDN in that it tries to put all the Google APIs under a single roof. The site consists of three main parts; information on Google APIs, links to projects Open Sourced by Google that are hosted on SourceForge and highlighted projects created by third parties that use Google's APIs.

The projects Open Sourced by Google are primarily test tools and data structures used internally. They are hosted on SourceForge although there seemed to be some dislike for the features of the site both from Chris and members of the audience. Chris did feel that among the various Open Source project hosting sites existing today, SourceForge was the one most likely to be around in 10 years. He mentioned that Google was ready to devote some resources to helping the SourceForge team improve their service.


 

Categories: Technology | Trip Report

These are my notes on Introduction to Yahoo! Search Web Services session by Jeremy D. Zawodny

The Yahoo! Search web services are available on the Yahoo! Search Developer Network(YSDN) site. YSDN launched by providing web services that allow applications to interact with local, news, Web, image and video search. Web services for interacting with Y!Q contextual search was launched during ETech.

Jeremy stated that the design goal for their web services was for them to be simple and have a low barrier to entry. It was hoped that this would help foster a community and create two-way communication between the Yahoo! Search team and developers.  To help foster this communication with developers YSDN provides documentation, an SDK, a blog, mailing lists and a wiki.

Requests coming in from client applications are processed by an XML proxy which then sends the queries to the internal Yahoo! servers and returns the results to developers. The XML proxy is written in PHP and is indicative of the trend to move all new development at Yahoo! to PHP.

Some of the challenges in building YSDN were deciding what communications features (wiki vs. mailing list), figuring licencing issues, and quotas on methods calls (currently 5,000 calls per day per IP address). In talking to Jeremy after the talk I pointed out that rate limiting by IP penalizes applications used behind a proxy server that make several requests a day such as RSS Bandit being used by Microsoft employees at work. There was a brief discussion about alternate approaches to identifying applications such as cookies or using a machine's MAC address but these all seemed to have issues.

Speaking of RSS, Jeremy mentioned that after they had implemented their Web services which returned a custom document format he realized that many people would want to be able to transform those results to RSS and subscribe to them. So he spoke to the developer responsible and he had RSS output working within 30 minutes. When asked why they didn't just use RSS as their output format instead of coming up with a new format, he responded that they didn't want to extend RSS but instead came up with their own format. Adam Bosworth mentioned afterwards that he thought that it was more disruptive to create a new format instead of reusing RSS and adding one or two extensions to meet their needs.

Then there was the inevitable REST vs. SOAP discussion. Yahoo! picked REST for their APIs because of its simplicity and low barrier to entry for developers on any platform. Jeremy said that the tipping point for him was when he attended a previous O'Reilly conference and Jeff Barr from Amazon stated that 20% of their API traffic was from SOAP requests but they accounted for 80% of their support calls.

Jeremy ended the talk by showing some sample applications that had been built on the Yahoo! Search web services and suggesting some ideas for members of the audience to try out on their own.


 

Categories: Trip Report | XML Web Services

These are my notes from the "Just" Use HTTP session by Sam Ruby

The slides for this presentation are available. No summary can do proper justice to this presentation so I'd suggest viewing the slides.

Sam's talk focuses on the various gotchas facing developers building applications using REST or Plain old XML over HTTP (POX). The top issues include unicode (both in URIs and XML), escaped HTML in XML content and QNames in XML content. A lot of these gotchas are due to specs containing inconsistencies with other specs or in some cases flat out contradictions. Sam felt that there is an onus on spec writers to accept the responsibility that they are responsible for interop and act accordingly.

At the end of the talk Sam suggested that people doing REST/POX would probably be better of using SOAP since toolkits took care of such issues for them. I found this amusing given that the previous talk was by Nelson Minar saying the exact opposite and suggesting that some people using SOAP should probably look at REST.

The one thing I did get out of both talks is that there currently isn't any good guidance on when to use SOAP+WSDL vs. when to use REST or POX in the industry. I see that Joshua Allen has a post entitled The War is Over (WS-* vs. POX/HTTP) which is a good start but needs fleshing out. I'll probably look at putting pen to paper about this in a few months.

 


 

Categories: Trip Report | XML Web Services

These are my notes on the Building a New Web Service at Google session by Nelson Minar

Nelson Minar gave a talk about the issues he encountered while shipping the Adwords API. The slides for the talk are available online. I found this talk the most useful of the conference given that within the next year or so I'll be going through the same thing at MSN.

The purpose of the Adwords API was to enable Google customers manage their adwords campaigns. In cases where users have large numbers of keywords, it begins to be difficult to manage an ad campaign using the Web interface that Google provides. The API exposes endpoints for campaign management, traffic estimation and reporting. Users have a quota on how many API calls they can make a month which is related to the size of their campaign. There are also some complex authentication requirements since certain customers give permission to third parties to manage their ad campaigns for them. Although the API has only been available for a couple of weeks there are already developers selling tools that built on the API.

The technologies used are SOAP, WSDL and SSL. The reason for using SOAP+WSDL was so that XML Web Service toolkits which perform object<->XML data binding could be used by client developers. Ideally developers would write code like

adwords = adwordsSvc.MakeProxy(...)
adwords.setMaxKeywordCPC(53843, "flowers", "$8.43")

without needing to know or understand a lick of XML. Another benefit of SOAP were that it has a standard mechanism for sending metadata (SOAP headers) and errors (SOAP faults).

The two primary ways of using SOAP are rpc/encoded and document/literal. The former treats SOAP as a protocol for transporting programming language method calls just like XML-RPC while the latter treats SOAP as a protocol for sending typed XML documents. According to Nelson, the industry had finally gotten around to figuring out how to interop using rpc/encoded due to the efforts of the SOAP Builders list only for rpc/encoded to fall out of favor and document/literal to become the new hotness.

The problem with document/literal uses of SOAP is that it encourages using the full expressivity of W3C XML Schema Definition (XSD) language. This is in direct contradiction with trying to use SOAP+WSDL for object<->XML mapping since XSD has a large number of concepts that have no analog in object oriented programming.

Languages like Python and PHP have poor to non-existent support for either WSDL or document/literal encoding in SOAP. His scorecard for various toolkits in this regard was

  • Good: .NET, Java (Axis)
  • OK: C++ (gSOAP), PERL (SOAP::Lite)
  • Poor: Python (SOAPpy, ZSI), PHP (many options)

He also gave an example that showed how even things that seemed fundamentally simple such as specifying that an integer element had no value could cause interoperability problems in various SOAP toolkits. Given the following choices

<foo xsi:nil="true"/>
<foo/>
nothing
<foo>-1</foo>

The first fails in current version of the .NET framework since it maps ints to System.Int32 which is value type meaning it can't be null. The second is invalid according to the rules of XSD since an integer cannot be empty string. The third works in general. The fourth is ghetto but is the least likely to cause problems if your application is coded to treat -1 as meaning the value is non-existent.

There are a number of other issues Nelson encountered with SOAP toolkits including

  • Nested complex types cause problems
  • Polymorphic objects  cause problems
  • Optional fields cause problems
  • Overloaded methods are forbidden.
  • xsi:type can cause breakage. Favor sending untyped documents instead.
  • WS-* is all over the map.
  • Document/literal support is weak in many languages.

Then came the REST vs. SOAP part of the discussion. To begin he defined what he called 'low REST' (i.e. Plain Old XML over HTTP or POX) and 'high REST'. Low REST implies using HTTP GETs for all API accesses but remembering that GET requests should not have side effects. High REST involves using the four main HTTP verbs (GET, POST, PUT, and DELETE) to manipulate resource representations, using XML documents as message payloads, putting metadata in HTTP headers and using URLs meaningfully.

Nelson also pointed out some limitations of REST from his perspective.

  • Development becomes burdensome if lots of interactivity in application (No one wants to write lots of XML parsing code)
  • PUT and DELETE are not implemented uniformly on all clients/web servers.
  • No standard application error mechanism (most REST and POX apps cook up their own XML error document format)
  • URLs have practical length limitations so one can't pass too muh data in a GET  
  • No WSDL, hence no data binding tools

He noted that for complex data, the XML is what really matters which is the same if you are using REST or SOAP’s document/literal model. In addition he felt that for read-only APIs, REST was a good choice. After the talk I asked if he thought the Google search APIs should have been REST instead of SOAP and he responded that in hindsight that would have been a better decision. However he doesn't think there have been many complaints about the SOAP API for Google search. He definitely felt that there was a need for more REST tools as well as best practices.

He also mentioned things that went right including :

  • Switch to document/literal
  • Stateless design
  • Having a developer reference guide
  • Developer tokens
  • Thorough interop testing
  • Beta period
  • Batch methods (every method worked with a single item or an array which lead to 25x speed up for some customers in certain cases). Dealing with errors in the middle of a batch operation become problematic though. 

There was also a list of things that went wrong 

  • The switch to document/literal cost a lot of time
  • Lack of a common data model
  • Dates and timezones (they allowed users to specify a date to perform operations but since dates don't have time zones depending on when the user sent the request the results may look like they came from the previous or next day) 
  • No gzip encoding
  • Having quotas caused customer confusion and anxiety  
  • No developer sandbox which meant developers had to test against real data
  • Using SSL meant that debugging SOAP is difficult since XML on the wire is encrypted (perhaps WS-Security is the answer but so far implementations are spotty)
  • HTTP+SSL is much slower than just HTTP.
  • Using plain text passwords in methods meant that users couldn't just cut & paste SOAP traces to the forum which led to inadvertent password leaks.

This was an awesome talk and I definitely took home some lessons which I plan to share with others at work. 


 

Categories: Trip Report | XML Web Services

These are my notes on the From the Labs: Google Labs session by Peter Norvig, Ph.D.

Peter started off by pointing out that since Google hires Ph.D's in their regular developer positions they often end up treating their developers as researchers while treating their researchers as developers.

Google's goal is to organize the world's data. Google researchers aid in this goal by helping Google add more data sources to their search engines. They have grown from just searching HTML pages on the Web to searching video files and even desktop search. They are also pushing the envelop when it comes to improving user interfaces for searching such as with Google Maps.

Google Suggest which provides autocomplete for the Google search box was written by a Google developer (not a researcher) using his 20% time. The Google Personalized site allows users to edit a profile which is used to weight search results when displaying them to the user. The example he showed was searching for the term 'vector' and then moving the slider on the page to showing more personalized results. Since his profile showed an interest in programming, results related to vector classes in C++ and Java were re-sorted to the top of the search results. I've heard Robert Scoble mention that he'd like to see search engines open up APIs that allow users to tweak search parameters in this way. I'm sure he'd like to give this a whirl. Finally he showed Google Sets which was the first project to show up on the Google Labs site. I remember trying this out when it first showed up and thinking it was magic. The coolest thing to try out is to give it three movies starring the same and watch it fill out the rest.


 

Categories: Technology | Trip Report

These are my notes on the From the Labs: Yahoo! Research Labs session by
Gary William Flake.

Yahoo! has two research sites, Yahoo! Next and Yahoo! Research Labs.  The Yahoo! Next site has links to betas of products that will eventually become products such as Y!Q contextual search, a Firefox version of the Yahoo! toolbar and Yahoo! movie recommendations.

The Yahoo! research team focuses on a number of core research areas areas including machine learning, collective intelligence, and text mining. They publish papers frequently related to these topics.

Their current primary research project is the Tech Buzz Game which is a fantasy prediction market for high-tech products, concepts, and trends. This is in the same vein as other fantasy prediction markets such as the Hollywood Stock Exchange and the Iowa Electronics Market. The project is being worked in in collaboration with O'Reilly Publishing. A product's buzz is a function of the volume of search queries for that term. People who constantly predict correctly can win more virtual money which they can use to bet more. The name of this kind of market is a dynamic peri-mutuel market.

The speaker felt Tech Buzz would revolutionize the way auctions were done. This seemed to be a very bold claim given that I'd never heard of it. Then again it isn't like I'm auction geek.


 

Categories: Technology | Trip Report

These are my notes on the From the Labs: Microsoft Research session by Richard F. Rashid, Ph.D.

Rick decided that in the spirit of ETech, he would focus on Microsoft Research projects that were unlikely to be productized in the conventional sense.

The first project he talked about was SenseCam. This could be considered by some to be the ultimate blogging tool. It records the user's experiences during the day by taking pictures, recording audio and even monitoring the temperature. There are some fancy tricks it has to do involving usage of internal motion detectors to determine when it is appropriate to take a picture so it doesn't end up blurry because the user was moving. There are currently 20 that have been built and clinical trials have begun to see if the SenseCam would be useful as aid to people with severe memory loss.

The second project he discussed was the surface computing project. The core idea around surface computing is turning everyday surfaces such as tabletops or walls into interactive input and/or display devices for computers. Projectors project displays on the surface and cameras detect when objects are placed on the surface which makes the display change accordingly. One video showed a bouncing ball projected on a table which recognized physical barriers such as the human hand when they were placed on the table. Physical objects placed on the table could also become digital objects. For example, placing a magazine on the table would make the computer copy it and when the magazine was removed a projected image of it would remain. This projected image of the magazine could then be interacted with such as by rotating and magnifying the image.

Finally he discussed how Microsoft Research was working with medical researchers looking for a cure for HIV infection. The primary problem with HIV is that it constantly mutates so the immune system and drugs cannot recognize all its forms to neutralize them in the body. This is similar to the spam problem where the rules for determining whether a piece of mail is junk mail keeps changing as spammers change their tactics. Anti-spam techniques have to use a number of pattern matching heuristics to figure out whether a piece of mail is spam or not. MSR is working with AIDS/HIV researchers to see whether such techniques couldn't be used to attack HIV in the human body.


 

Categories: Technology | Trip Report

These are my notes in the Vertical Search and A9 by Jeff Bezos.

The core idea behind this talk was powerful yet simple.

Jeff Bezos started of by talking about vertical search. In certain cases, specialized search engines can provide better results than generic search engines. One example is searching Google for Vioxx and performing the same search on a medical search engine such as PubMed. The former returns results that are mainly about class action lawsuits while the latter returns links to various medical publications about Vioxx. For certain users, the Google results are what they are looking for and for others the PubMed results would be considered more relevant.

Currently at A9.com, they give users the ability to search both generic search engines like Google as well as vertical search engines. The choice of search engines is currently small but they'd like to see users have the choice of building a search homepage that could pull results from thousands of search engines. Users should be able to add any search engine they want to their A9 page and have those results display in A9 alongside Google or Amazon search results. To facilitate this, they now support displaying search results from any search engine that can provide search results as RSS. A number of search engines already do this such as MSN Search and Feedster. There are some extensions they have made to RSS to support providing search results in RSS feeds. From where I was standing some of the extension elements I saw include startIndex, resultsPerPage and totalResults.

Amazon is calling this initiative OpenSearch.

I was totally blown away by this talk when I attended it yesterday. This technology has lots of potential especially since it doesn't seem tied to Amazon in any way so MSN, Yahoo or Google could implement it as well. However there are a number of practical issues to consider. Most search engines make money from ads on their site so creating a mechanism where other sites can repurpose their results would run counter to their business model especially if this was being done by a commercial interest like Amazon.


 

Categories: Technology | Trip Report

These are my notes on the Remixing Technology at Applied Minds session by W. Daniel Hillis.

This was a presentation by one of the key folks at Applied Minds. It seems they dabble in everything from software to robots. There was an initial demo showing a small crawling robot where he explained that they discovered that six legged robots were more stable than those with four legs. Since this wasn't about software I lost interest for several minutes but did hear the audience clap once or twice for the gadgets he showed.

Towards the end the speaker started talking about an open market place of ideas. The specific scenario he described was the ability to pull up a map and have people's opinions of various places on the map show up overlayed on the map. Given that people are already providing these opinions on the Web today for free, there isn't a need to have to go through some licenced database of reviews to get this information. The ability to harness the collective consciousness of the World Wide Web in this manner was the promise of the Semantic Web which the speaker felt was going to be delivered. His talk reminded me a lot of the Committee of Gossips vision of the Semantic Web that Joshua Allen continues to evangelize.

It seems lots of smart people are getting the same ideas about what the Semantic Web should be. Unfortunately, they'll probably have to route around the W3C crowd if they ever want to realize this this vision.


 

Categories: Technology | Trip Report