Archive for the 'ATOM' Category

REST Redux

The Rise of REST! I’ve noticed an increasing number of “What is REST?”, or alternatively, explanations of REST appearing lately. Most attempt to compare REST to SOAP which is basically the wrong argument. SOAP and REST cannot be directly compared, though for the last seven years it has been. Keep in mind that REST does not mean exclusively HTTP. Just take a look at the Java content-management standards JSR 170 and JSR 283 for an example of an API based on REST using Java.

Before I attempt to put my own spin on what and why you would use REST; If you are interested at all on Software Architecture, then I highly recommend that you take the time to read Roy’s dissertation – maybe even twice. It is a tour-de-force of what Software Architecture is. And if you feel really brave read Rohit’s companion dissertation.

Why base your software architecture on REST?

The high-level answer: When you don’t “own” each component along the wire it is important that each component be allowed to evolve independently. What matters in this context is the ability to work with components at different levels of implementation. That means constraining the interface to a commonly agreed standard so every component along the wire can understand what it means to GET, PUT, POST, DELETE etc.

Secondly no matter how fast the wire becomes we are still constrained by the speed of light i.e. there will always be latency. To combat this you end up using a course grained Document-Centric mode of interaction – sending HTML/XML/JSON documents etc – instead of a fine grained Control-Centric mode. The latter is RPC the former REST.

To make sure that each component knows what is being talked about it needs to be identified. Hence URIs identify a Resource. Response meta-data describes the context of the document that was returned. [in REST terms the context of the Representation returned.]

A Resource can be thought of as an object that implements various interfaces. It is a concept, a notional software component that implements many interfaces. In REST those interfaces are identified by URIs.

Without constrained interfaces, a data-centric mode of interaction and identity, caches won’t work for example. This also means that Hypertext now becomes the engine of application state. Every component interface implements a state machine regardless of the underlining component technology, hence in REST URI become a way of making the application state explicit. Tim Ewald framed it nicely:

“The essence of REST is to make the states of the protocol explicit and addressible by URIs. The current state of the protocol state machine is represented by the URI you just operated on and the state representation you retrieved. You change state by operating on the URI of the state you’re moving to, making that your new state. A state’s representation includes the links (arcs in the graph) to the other states that you can move to from the current state. This is exactly how browser based apps work, and there is no reason that your app’s protocol can’t work that way too. (The ATOM Publishing protocol is the canonical example, though its easy to think that its about entities, not a state machine.)”

RPC (and therefore SOAP) rely on tool support to be vaguely usable. REST works because humans can do a wget and see what comes back. That reduces the coordination costs of building systems.

If you control all the components in the chain and you release them all at the same time and there is no latency issue to worry about, use a RPC style of interaction. Otherwise use REST. In today’s Web2.0 climate that means using a REST style, as a minimum, for externally exposed Resources via APIs.

An Architectural Style

REST is an Architectural Style in the true sense of Alexander’s patterns, compared to, the more prescriptive approach adopted by the Design Patterns movement, lead by the Gang of Four. SOAP is a XML language for specifying and constructing messages for a network protocol. The typical message exchange pattern is modelled after the RPC style. So just to confuse the matter SOAP can be compared more directly to HTTP; the latter being a Network API that attempts to conform to the REST style.

If REST is to be compared then it should be compared to RPC or SOA. Both are approaches to software architecture. SOAP is an implementation that can be used in either a Control-centric (RPC/SOA) style or a Document-centric (REST) style. It just happens than most uses of SOAP are RPCs.

No comments

The next thing after 2.0!

It seems technology suffers from fashion as much as the fashion industry in that we must keep moving on to the next big thing before we have even understood the current thing. Regardless, despite disliking the term and being too lazy to even attempt defining a new one, recently I took the bait over at ReadWriteWeb to provide a definition for Web3.0; It seems they liked my definition of Web3.0. Here is what I wrote:

Web 1.0 – Centralised Them.
Web 2.0 – Distributed Us.
Web 3.0 – Decentralised Me

Hindsight: Web 1.0 turned into a broadcast medium. It was all about them. A case of industrial age thinking applied to a new landscape. Web 2.0, largely based on an analysis of what worked in Web1.0, is an alignment with TBL’s initial vision of the Web. The Web as connective tissue between us. Platform, participation and conversation. Really it is more than the Web. It is the Internet. It is new practices too. Ultimately it is about connectivity; applying constrains in the form of some sort-of agreed upon standards that make it easier to talk to one another. With new layers of connective wealth come new tools. In Web2.0’s case that allowed new forms of communication. With it associated ‘acceptable’ business models – hence the Google economy.

Web 1.0 was the first time to show the value of standards, Web 2.0 is teaching us how liberating standards can be. Web 3.0 will reflect on what worked in Web2.0. It will mean more constraints for better communication/connectivity. Improved connectivity will mean revised practice and new business models.

Therefore Web 3.0 must be about me! It’s about me when I don’t want to participate in the world. It’s about me when I want to have more control of my environment particularly who I let in. When my attention is stretched who/what do I pay attention to and who do I let pay attention to me. It is more effective communication for me!

When it is about me it means Web 3.0 must be about more semantics in information, but not just anything. Better communication comes from constraints in the vocabularies we use. Micro formats will lead here helping us to understand RDF and the Semantic Web. With more concern over my attention comes a need to manage the flow of information. This is about pushing and pulling information into a flow that accounts for time and context. Market based reputation models applied to information flows become important. Quality of Service (QOS) at the application and economic layer where agents monitor, discover, filter and direct flows on information for me to the devices and front-ends that I use. The very notion of application [Application is a very stand-alone PC world-view. Forget the Web, Desktop, Offline/Online arguments] disappears into a notion of components linked by information flows. Atom, the Atom API and semantics, particularly Micro formats initially, are the constraints that will make this happen. Atom features not because of technical merit but by virtue of it’s existing market deployment in a space that most EAI players won’t even consider a market opportunity. Hence Web based components start using Atom API as the dominate Web API – Feed remixing is indicative. Atom will supplant WS* SOA.

User centric identity takes hold. This extends the idea that everyone has an email address and mobile number, why not manage it for single sign-on and more. Universal Address-book anyone?

More Market based brokerage business models emerge, earning revenue on the ‘turn’, as we learn more about the true power of AdSense/Adword’s underling business model and realise there are close parallels to the worlds financial markets.

Reliable vocabularies, user identity and trusted [i.e. user controllable] reputation models, market based brokerage business models all become a necessity as the more decentralized event driven web becomes a reality.

Web 3.0 – a decentralized asynchronous me.

There were a few things I forgot to put into the above definition and from the comments a few things that need explanation. I’ll attempt to expand on the above in latter posts as I’m a little stuck for time.

What I left off is the relationship to the physical world; “the Internet of Things” with 2D Bar Codes, RFID etc. and Just-in-time just-about-one-to-one manufacturing that is partly represented by what Threadless and others are doing. I’ll also need to clarify what I mean by Them, Us, Me. And why Web 3.0 cannot be classified as “Read / Write / Execute.”

Some comments past on to me ask how is this different from what Web2.0 is about? At a technology level it really isn’t, the technology is already here. From a cultural and hence practice level it is. As we starting seeing more value in using things like Atom, Meta-Data, Open-Data and feed remixing etc, then how we use the Internet and our connected devices will change. That is what, at the core, is the basis of Web2.0 – changing usage and practice.


Scaling Yahoo! Pipes

Yahoo! Pipes are CloggedIf you’ve been trying to access Yahoo! Pipes and come up with a request for Mario then have some sympathy for Yahoo! as Dare Obasanjo comments.

As someone who works on large scale online services for a living, Yahoo! Pipes seems like a scary proposition.

Basically the service is likely to have hit the twin sisters of scaling large systems, CPU and I/O bounds.The nature and flexibility that Yahoo! Pipes provides: User defined queries over changing data streams that are activated every time a pipe is “pulled” will create a heavy load. Dare continues.

It combines providing a service that is known for causing scale issues due to heavy I/O requirements (i.e. serving RSS feeds) with one that is known for scaling issues due to heavy CPU and I/O requirements (i.e. user-defined queries over rapidly changing data). I suspect that this combination of features makes Yahoo! Pipes resistant to popular caching techniques especially if the screenshot below is any indication of the amount of flexibility [and thus processing power required] that is given to users in creating queries.

This has always been an issue if you consider a centralised event routing infrastructure. I suspect it is made worse by the “pull” nature of feeds. Even if if-modified headers or etags are used in the HTTP request. To determine if a user-defined feed has changed it would require processing the whole chain effectively adding a whole lot of processing over head. This is however a naive way of approaching the problem. An internal event architecture and caching are applicable. The caching is just different to the way content caching currently works. However the answer, I suspect, is not to centralise. The general infrastructure represented by Yahoo! Pipes should be deployed by millions. I have my own router for IP traffic so why not have my own router for application data traffic?

No comments

Yahoo Pipes a Cloud based Feed Router

Yahoo Pipes was released yesterday (last night NZ Time). What Yahoo Pipes represents is the next stage in using the Web as a platform. That is: RSS and Atom are becoming the connective tissue of the web and web applications the components. This was the basis of my Talk at Kiwi Foo – “Replacing my Wife with Atom.*” I commented I was surprised this [general routing infrastructure for Atom, not the replacement of my Wife] had not been done yet, after all the ideas of routing and remixing feeds have been discussed for a long time in the early history of RSS. Before that if you consider RSS/ATOM and the infrastructure around it as a Universal Event Bus. As Jeremy Zawodny states “For far too long now RSS has been used in ways that don’t really tap its true potential.” Well now Yahoo is making a start in the right direction.

Unfortunately I can’t get in, just getting “Our Pipes are Clogged!” – missed opportunity or what? Well it shows, to some degree that at least, the info junkies want this sort of thing. As a result I can’t really comment too much on the service itself, but I can comment on the underlining ideas – this was after all the motivation for re-entering the blogsphere.

[Tim O’Reilly has a good write-up as does ReadWriteWeb and Anil Dash. While Brady Forrest looks a little closer. ]

The basis of the talk I gave at Kiwi Foo was this:

Atom and the Atom publishing protocol will disrupt the way ‘network’ applications are integrated and become the way Internet scale Pub+Sub and routable applications architectures will evolve…Therefore make your Web API’s atom based to make integration easier – Look Ma no code!

While to some degree the arguments apply to RSS, ATOM [IMHO] has the advantage in the longer term. The arguments are not new. Rohit Khare has been thinking about this stuff for a long time. He gave a talk about SOAP routing at ECON 2002. I just applied similar arguments to ATOM but tried to include the human component – conceptually feeds are more human friendly than SOAP ever will be. That observation then naturally leads into a Disruptive Technology argument w.r.t. EAI. Just take a look at the dabbleDB demo that Avi Bryant put together at FOO last year, or RSSBus for examples. (there are several other examples that I’ll leave for now.)

An observation I attempted to justify was: prior technologies have attempted to implement Internet Scale Pub+Sub but failed not because the technologies weren’t up to it (they may not have been) but that as a broader community we still haven’t grokked what the Web as a platform really means. As a result we haven’t understood what integrating applications beyond the Point-2-Point mashup type actually involves. Oddly enough though we are use to solving these integration issues at the network-level. Just look at the Internet and TCP/IP. The same principles of message (packet) based approaches apply in the Enterprise as any EAI vendor will tell you. ATOM is approaching the EAI issue from the grass-roots up. Unlike SOAP that was top down even though it professed not to be. More importantly for this sort of service composition to really take off Atom and the Web needs a “joe average” programming model.

Yahoo Pipes is a giant leap in that direction for the Web even though we have been programming the Web, for human consumption, for sometime now. Every time you add a feed URL to a feed reader or personalised home page you are connecting disparate web services together, you are programming. Labview has been doing this sort of thing for a long time. Lego RoboLab and the Lego Robotics programming environment, Microsoft Robotics Studio Visual programming are other great examples. Ben Goodger recently pointed me to another example on OSX, Automator.

Does this mean the programming model for the Web platform is a visual wiring diagram. Why not? Nor does it have to be provided as a cloud based service like Pipes. Since Pipes produces feeds these can then be glued to an instance of Pipes (RSSBus/Automator) running inside my firewall, on my desktop, or in my Browser making the offline and online divide seamless – portable ‘backend’ widgets for the Web (more on this later). Aside from all this, Pipes adds an interesting twist to information personalisation.
* [in case you are wondering the title of my talk refers to a user scenario managing and filtering many information sources and notifying me of things that are important at a given time or place using the most appropriate delivery. A job my partner, Lynne, currently does to some extent and Gmail does for my email.]

No comments

Call it a new years resolution.

After a five year hiatus with what was at best a half-hearted attempt at public blogging using Radio Userland I’ve decided I’ll give it another go. I’ll try to stick to a regular posting routine – at least three times a week – and see what materialises from it.

More than anything I’ll use this site as ramblings for thoughts on software development that interest me and to keep notes related to my digging and tinkering on the Web. I’ll mostly be focusing on Internet Scale Architecture and highly concurrent system design and development. I have a particular interest in event-based routing architectures and ATOM in particular.

I owe some thanks for motivating me into restarting my blog to Russ Weakly. After patiently listening to my ideas on RSS/ATOM, microformats and information brokerage, over dinner one evening late last year, suggested I start writing about some of them. So here it goes.

1 comment