A couple of blogs I'm subscribed to are pimping the brand new Yahoo! Pipes which I unfortunately can't seem to access right now. You can read some of the hype in blog posts like Jeremy Zawodny's Yahoo! Pipes: Unlocking the Data Web and Tim O'Reilly's Pipes and Filters for the Internet where it is described as "milestone in the history of the internet". I'd have loved to try out the service giving my interest in mashups and feed syndication but the site seems to be down or is just really, really slow.

As Dave Winer writes in his post Pipes Investigation

I see that Yahoo has a new web app, called Pipes, that looks to me like a feed construction kit. It takes RSS inputs, processes them in ways that are specified by the user, and produces feeds as its output.
From a quick persual of the functionality last night and the fact that the server isn't responding right now (5:45AM Pacific), it seems this app uses lots of CPU on the server

As someone who works on large scale online services for a living, Yahoo! Pipes seems like a scary proposition. It combines providing a service that is known for causing scale issues due to heavy I/O requirements (i.e. serving RSS feeds) with one that is known for scaling issues due to heavy CPU and I/O requirements (i.e. user-defined queries over rapidly changing data). I suspect that this combination of features makes Yahoo! Pipes resistant to popular caching techniques especially if the screenshot below is any indication of the amount of flexibility [and thus processing power required] that is given to users in creating queries.

Really interesting idea though. I agree with Dave Winer that this is definitely fodder for geeks and not the average Web user. After all, RSS still hasn't crossed the adoption chasm with average Web users let alone an RSS feed remixing service.


Thursday, February 8, 2007 5:46:44 PM (GMT Standard Time, UTC+00:00)
(I haven't been able to get to it either, but...) I'd suspect that they have some sort of intelligent caching going on under the hood -- like only regenerating the output feed if atleast one of the input feeds has been updated. Or maybe a max regen's per minute/hour or something. I'd think it'd be less-than-scalable (i.e., server murder) regenerating output feeds on *every* request.
Thursday, February 8, 2007 6:35:06 PM (GMT Standard Time, UTC+00:00)
It is very RSS-workflow tools, but you could certainly build up some interesting queries *that you can share*. Its a shame the basic sources are so limited.

I tried to hook up a gmail atom feed to the babelfish en to fr translation, to get my unread email in french. No joy, as it didnt support basic auth on a url. pity.

I could imagine writing atom/RSS feeds just for this app though, using it for the post processing.

I wonder what the business model will be.
Friday, February 9, 2007 6:18:19 AM (GMT Standard Time, UTC+00:00)
Take a look at a superset of Yahoo pipes that has been in the market for over a year now, Proto (http://www.protosw.com)
Friday, February 9, 2007 7:44:53 AM (GMT Standard Time, UTC+00:00)
"definitely fodder for geeks and not the average Web user"

I couldn't agree more, built by geeks for geeks :) I had no idea so many people would show up today.
It's still a very early beta so just bear with us. Would like to hear your thoughts after you get a chance to play with it more.

Comments are closed.