Yahoo Pipes a Cloud based Feed Router

Yahoo Pipes was released yesterday (last night NZ Time). What Yahoo Pipes represents is the next stage in using the Web as a platform. That is: RSS and Atom are becoming the connective tissue of the web and web applications the components. This was the basis of my Talk at Kiwi Foo – “Replacing my Wife with Atom.*” I commented I was surprised this [general routing infrastructure for Atom, not the replacement of my Wife] had not been done yet, after all the ideas of routing and remixing feeds have been discussed for a long time in the early history of RSS. Before that if you consider RSS/ATOM and the infrastructure around it as a Universal Event Bus. As Jeremy Zawodny states “For far too long now RSS has been used in ways that don’t really tap its true potential.” Well now Yahoo is making a start in the right direction.

Unfortunately I can’t get in, just getting “Our Pipes are Clogged!” – missed opportunity or what? Well it shows, to some degree that at least, the info junkies want this sort of thing. As a result I can’t really comment too much on the service itself, but I can comment on the underlining ideas – this was after all the motivation for re-entering the blogsphere.

[Tim O'Reilly has a good write-up as does ReadWriteWeb and Anil Dash. While Brady Forrest looks a little closer. ]

The basis of the talk I gave at Kiwi Foo was this:

Atom and the Atom publishing protocol will disrupt the way ‘network’ applications are integrated and become the way Internet scale Pub+Sub and routable applications architectures will evolve…Therefore make your Web API’s atom based to make integration easier – Look Ma no code!

While to some degree the arguments apply to RSS, ATOM [IMHO] has the advantage in the longer term. The arguments are not new. Rohit Khare has been thinking about this stuff for a long time. He gave a talk about SOAP routing at ECON 2002. I just applied similar arguments to ATOM but tried to include the human component – conceptually feeds are more human friendly than SOAP ever will be. That observation then naturally leads into a Disruptive Technology argument w.r.t. EAI. Just take a look at the dabbleDB demo that Avi Bryant put together at FOO last year, or RSSBus for examples. (there are several other examples that I’ll leave for now.)

An observation I attempted to justify was: prior technologies have attempted to implement Internet Scale Pub+Sub but failed not because the technologies weren’t up to it (they may not have been) but that as a broader community we still haven’t grokked what the Web as a platform really means. As a result we haven’t understood what integrating applications beyond the Point-2-Point mashup type actually involves. Oddly enough though we are use to solving these integration issues at the network-level. Just look at the Internet and TCP/IP. The same principles of message (packet) based approaches apply in the Enterprise as any EAI vendor will tell you. ATOM is approaching the EAI issue from the grass-roots up. Unlike SOAP that was top down even though it professed not to be. More importantly for this sort of service composition to really take off Atom and the Web needs a “joe average” programming model.

Yahoo Pipes is a giant leap in that direction for the Web even though we have been programming the Web, for human consumption, for sometime now. Every time you add a feed URL to a feed reader or personalised home page you are connecting disparate web services together, you are programming. Labview has been doing this sort of thing for a long time. Lego RoboLab and the Lego Robotics programming environment, Microsoft Robotics Studio Visual programming are other great examples. Ben Goodger recently pointed me to another example on OSX, Automator.

Does this mean the programming model for the Web platform is a visual wiring diagram. Why not? Nor does it have to be provided as a cloud based service like Pipes. Since Pipes produces feeds these can then be glued to an instance of Pipes (RSSBus/Automator) running inside my firewall, on my desktop, or in my Browser making the offline and online divide seamless – portable ‘backend’ widgets for the Web (more on this later). Aside from all this, Pipes adds an interesting twist to information personalisation.
* [in case you are wondering the title of my talk refers to a user scenario managing and filtering many information sources and notifying me of things that are important at a given time or place using the most appropriate delivery. A job my partner, Lynne, currently does to some extent and Gmail does for my email.]

No comments Digg this

No comments yet. Be the first.

Leave a reply