REST (1): misunderstood and important, how and why

I want to take a break from declarative/curly posts and write a bit about REST. I hope this will help spawn some discussion among developers I work with and know.
I am on last pages of (the rightfully acclaimed) RESTful web services, by Richardson and Ruby book.  The reading felt a bit repetitive, it felt like hammering of ideas into my head, but they probably needed hammering.  Overall a great book, one that opened my mind to what REST is about (or should be about).

Somewhere half way through the book things started to click.   (I am trying to share some of these clicks in this and in a next few posts.)  I read quite a bit about REST before reaching for the book.  The first question is why did it take so long to click?  

I came out of this reading with a much better understanding of the HTTP protocol itself, so this must be a part of the learning curve.  But, I also blame my slowness on two things: first reason is the amount of confusion surrounding REST,  second reason is harder to explain:  REST is somewhat complex if you look at it as a set of implementation guidelines (the how),  it is also quite straightforward if you look at it as what it is for (the why).   It is hard to understand the how until you get the why.

Developers like to focus on the mechanics of things (the how).  Developers also have deadlines and work in a task-oriented environment where there maybe many whys but the goals of Uniform Interface, ease of future integration and benefits of semantic linking are probably not likely to be among the whys which pay their paychecks.  REST become such a buzzword that it ends up being used even if the why is not on the radar.  REST gets half-implemented but holds on to its full name. This fuels confusion.

REST Confused: 

The term REST is used a lot these days, I hear it at work,  I see a lot about it on the web.  Many developers are very interested in it and know a lot about it (way more than I do).  Last year's Uberconf REST offerings were very well attended.  At the same time, there is quite a bit of misunderstanding and confusion about what REST is.  (I was among the confused before reading this book.)  And I think that the industry-wide confusion on this topic will be winning.
The book authors decided to stay away from the REST term most of the time and have used the term ROA (Resource Oriented) to avoid the confusion.  (I stay with the terms REST and RESTful.)   
The book also introduced the therm REST-RPC Hybrid to describe many existing web services and to differentiate them from the true REST.  There is quite a bit of wit in this self-contradicting term.

It is just like Object Oriented
A somewhat forgotten old paper by Tim Rentsch (back in 1982) included a prophecy about what OO will become:  "Every manufacturer will promote his products as supporting it. Every manager will pay lip service to it. Every programmer will practice it (differently). And no one will know just what it is."

One more old OO related quote stays in my brain and does not want to leave:  I was at some sort of a conference and talking to vendors in the booth area.  One of them (he knew he is talking to a developer) was doing his spiel and said this: '... and we will be object oriented by the end of the next quarter'.   I am not trying to contribute ideas for a next Dilbert episode.  The scary thing is: that does not sound like something the marketing group would came up with on their own. I fear it came from the developers. 

I think REST became like OO from these last 2 quotes. Part of the problem is that, like OO, everyone wants to claim it, yet it is not so easy to implement (my next posts will talk more why it is hard) but the lip service is easy so the term is used a lot.

Here is a quote from the above book: "Both REST and web services have become buzzwords. They are chic and fashionable. These terms are artfully woven into PowerPoint presentations by people who have no real understanding of the subject".

Side NOTE:  a similarity between REST and OO:  both are often used as check boxes and both are really progress bars.

THE WHY:
I believe the fact that the term REST is misused and misunderstood causes lack of its true adoption.  We are missing out on some amazing opportunities.  REST could be a big step towards fulfilling a fantasy in which computer programs can surf the web the way humans do today.

Why REST is such a good idea?:  REST is the opposite of API.  APIs are all different, REST services are trying to be all the same.  

Let me quote the book:

"One alternative to explaining everything is to make your service look like other services.  If all services exposed the same representation formats, and mapped URIs to resources in the same way... well, we can't get rid of client programming altogether, but clients could work at higher level than HTTP."

I came up with this imprecise, sort-of definition of REST focusing on the why:  You wrote a very RESTful app if you have exposed your app functionality to the programmable web without API specification and without ambiguity.

I think the problem is that people think that REST is all about using GET, PUT, POST, DELETE (maybe OPTIONS, HEAD), some will know that POST as it stands today is better not used, but that is where it often ends.  We think of the mechanics and loose the big picture.  The net result is that we use the term REST a lot,  but what we develop typically has nothing to do with being easily accessible by the 'rest' of the programmable web.   

Example:

Say, you are developing a web application serving web pages and need to expose Ajax data from server to your Java Script library in a few places (just a GET request for simplicity). 

JSON is not very RESTful (point one: developers should be aware of the fact) but, if you decide on using JSON, still ask yourself these questions: 

• How cryptic is the URL, does it look anything like a location of a resource?  Any database IDs in it? (Would you expect external REST client to know your database IDs?)
• Can someone not very familiar with this project get any information from your JSON? 
• How much is this JSON specific to the current version of the UI:
• Is it just the fields that you need or a more meaningful representation? 
• Are you pre-formatting the data for your UI on the server?
• How is your error handling and status codes?  (say, you listing employees in a department and someone will issue a URL GET with invalid department: will your service just return 500 with the error message in plain text? - hey you know your JavaScript would never do that so why care about correct status codes or consistent content type?...).
• And finally how can another developer discover this URL?  Do you expect them to simply construct the URLs?

Lots of REST is simply common sense if you understand the why.

We are missing on big opportunities:

The first reason is so other developers can access/integrate easily with your application. 

The second reason is so that the future semantic web will find the data exposed by your app.

Such goal should be very important for businesses which want their products to be 'known' or educational institutions, organizations defining standards, many government institutions which want their data to be transparent and available to programs at large who may want to use it, study it, analyze it. 

These 2 reasons should create a big push for writing RESTful apps, but the reality is the exact opposite. Today businesses run hundreds of applications, these applications do not know how talk to each other.  Any integration is complex and expensive.  New applications are being developed (I participated in developing some of them) and REST is last thing on anyone's mind. 

The targets of semantic web today are not government, science, or even manufactured goods (although, Best Buy made recent news to the contrary), but tweeter, facebook, blogger, etc.  Who likes whom on facebook will benefit from all of this,  but is there any semantic web development for, say, endangered species?  This is not to criticize social web, kudos to them for innovating, this is to criticize everyone else who should be part of this and decides to stay behind.

Next post will have more mechanics:

I think a big part of the problem is that good REST is not so easy to implement. If it is hard and it is a buzzword people are bound to overuse it.  Buzzwords loose their meaning very fast, just like OO term did.

My next post will be about what is so hard about REST.