These are my notes on Introduction to Yahoo! Search Web Services session by Jeremy D. Zawodny

The Yahoo! Search web services are available on the Yahoo! Search Developer Network(YSDN) site. YSDN launched by providing web services that allow applications to interact with local, news, Web, image and video search. Web services for interacting with Y!Q contextual search was launched during ETech.

Jeremy stated that the design goal for their web services was for them to be simple and have a low barrier to entry. It was hoped that this would help foster a community and create two-way communication between the Yahoo! Search team and developers.  To help foster this communication with developers YSDN provides documentation, an SDK, a blog, mailing lists and a wiki.

Requests coming in from client applications are processed by an XML proxy which then sends the queries to the internal Yahoo! servers and returns the results to developers. The XML proxy is written in PHP and is indicative of the trend to move all new development at Yahoo! to PHP.

Some of the challenges in building YSDN were deciding what communications features (wiki vs. mailing list), figuring licencing issues, and quotas on methods calls (currently 5,000 calls per day per IP address). In talking to Jeremy after the talk I pointed out that rate limiting by IP penalizes applications used behind a proxy server that make several requests a day such as RSS Bandit being used by Microsoft employees at work. There was a brief discussion about alternate approaches to identifying applications such as cookies or using a machine's MAC address but these all seemed to have issues.

Speaking of RSS, Jeremy mentioned that after they had implemented their Web services which returned a custom document format he realized that many people would want to be able to transform those results to RSS and subscribe to them. So he spoke to the developer responsible and he had RSS output working within 30 minutes. When asked why they didn't just use RSS as their output format instead of coming up with a new format, he responded that they didn't want to extend RSS but instead came up with their own format. Adam Bosworth mentioned afterwards that he thought that it was more disruptive to create a new format instead of reusing RSS and adding one or two extensions to meet their needs.

Then there was the inevitable REST vs. SOAP discussion. Yahoo! picked REST for their APIs because of its simplicity and low barrier to entry for developers on any platform. Jeremy said that the tipping point for him was when he attended a previous O'Reilly conference and Jeff Barr from Amazon stated that 20% of their API traffic was from SOAP requests but they accounted for 80% of their support calls.

Jeremy ended the talk by showing some sample applications that had been built on the Yahoo! Search web services and suggesting some ideas for members of the audience to try out on their own.


Friday, March 18, 2005 6:26:23 PM (GMT Standard Time, UTC+00:00)
This is frustrating. If the industry does not learn to evolve the way it talks about Web services, the debate about REST vs. WS-* will never advance to a useful level. It's not REST vs. SOAP. It's REST vs. RPC-Style Web services that use SOAP. There is a big difference.
Friday, March 18, 2005 7:26:55 PM (GMT Standard Time, UTC+00:00)
Actually in practice it is Plain Old XML (POX) vs. SOAP. Most services on the Web that claim to be RESTful (e.g. Bloglines,, Flickr, etc) actually are not. Instead they are POX services.

Whether it is RPC-style SOAP or message oriented SOAP is irrelevant. The bottom line in practice is that the overhead of SOAP and the fact that it ignores a fundamental aspect of the Web (use URIs for identifying resources) means that it typically can't compete with REST/POX when one wants to build a 'reach' service on the Web.

I believe the only question left up for debate is whether SOAP should be used for building 'rich' services on the Web. But with the growing popularity of AJAX applications this isn't even a slam dunk.
Comments are closed.