REST (2): why NOT easy

Continuation of the previous post motivated by my reading of the RESTful Web Services book.

This post is a high level overview of the topic.  In my next posts I will provide more concrete examples of why REST is not so easy.  I am talking about REST in the context of HTTP protocol only.  I assume the reader has basic familiarity what are the 6 HTTP verbs (TRACE and CONNECT do not count ;) and what REST is or at least what is typically painted to be.

When you develop RESTful web services you will be dealing with

• Resources (think: flight reservation as a resource example)
• URLs (which are the resource addresses, think:  http://.../{userName}/travel/reservations/flights/{tripName})
• Resource Representations (for example some XML vocabulary,  think: XHTML document describing flight reservation, seat assignments, etc)

Thus, designing your web service application you will need to think about:

• How to define URLs (some ways are more RESTful than others)  
• What standards to use to define resource representations (some standards are more RESTful than others)

In addition, the book identifies following properties which make things RESTful (or as the book calls them ROA-Resource Oriented): 

• Addressability (measure of how well you can access your resources with URLs, ideally each resource should have a URL) 
• Uniform Interface (requires that the web service exposes subset of the 6 HTTP verbs and uses HTTP protocol itself to manage resources,  think: GET gets current flight reservation, PUT makes changes to it, DELETE deletes it,  think of using HTTP header and status codes) 
• Statelessness (no state on the server, state is in hypermedia)
• Connectedness (measure how well are your resource representations linked,  think: link or a tags linking to user account with list of other reservations this user has made, link to the airline flight itself, link or form to a ticket resource (to buy the ticket for this reservation), maybe a form tag defining what can be changed using PUT, etc) 

The first expense of adding REST to your project will be some design time:  (1) split your application into many resources, (2) create URLs for all the resources,  (3) map your problem domain to the Uniform Interface (that includes using standard status codes,  user of more advanced HTTP header stuff),  (4) design your representations (make formatting decisions), (5) connect the resources with some type of hypermedia.

There are 4 big problems categories which make it hard:

PROBLEM 1:  Tool Support: 

Most tools including Grails or Rails are designed to do something else with ease (to quickly create web apps) and the simplifying decisions which made them successful also makes doing other things (like creating a RESTful web service) hard.  The book lamented about Rails (Grails) has not been covered.

If you are using tools designed for REST (like restlet) this point does not apply.

PROBLEM 2:  Finding Standard Vocabulary for Resource Representation:
Some form of XML seem to be best bet, but a vocabulary designed for your problem domain simply may not exist.  Using XHTML maybe a good option.  There may be some relevant microformats for your problem domain.

PROBLEM 3:  How to implement Connectedness: 

There are very few tools or even standards which could help you to connect your resources.  Connectedness may be the most important part of being RESTful and is the hardest to implement.  The point is that without it REST it is just another API only with fewer verbs and many, many nouns. Sure there is a benefit of not needing to know which verb to use (you have to when using APIs), but there is a tradeoff of having to know many nouns (URLs).   Connectedness implies that REST client can discover the URLs and it does not need to know them and should not be constructing them!

Here is where semantic web becomes important.  Things like microformats, RDF, etc.
XHTML has link and a tags and forms (HTML 5 forms get better).  Atom feeds/APP, etc also define cross linking between resources.  Microformats play a big role in qualifying the 'connections' between resources. 

Future RESTful clients will be able to semantically know the links.

HTTP protocol itself can help in connectedness a bit with its location header and various status codes: 201 (Created), and many in the 3xx (Redirection) range.  But mostly connecting resources will be the job of your resource representations.  No free lunch here. To connect your resources some more serious work needs to be done and you will be looking at exposing you data trough things like Atom feeds.

PROBLEM 4:  Confusion: 

Remember the big picture from my last post?:  APIs are all different, REST is trying to be the same.  The uniformity will not happen if everyone is confused about what REST is, yet claims to support it.

REST is often understood as simply some level of support for the 4 verbs in HTTP (GET, POST, PUT and DELETE).  I maybe repeated myself here, but this trivialized view is demonstrated by most frameworks (including Ext JS, Rails, Grails). Adopting this simplified view point creates confusion and probably impedes support for more meaningful REST functionality these frameworks could provide.

Example:  RESTful JSON. JSON is not connectable (at least not in any standard way), so how can web service be truly RESTful if it is serving JSON only?

This was my high level overview.  Next post will deal with more concrete examples of why it is hard to REST (and maybe easier to stay BUSY ;)

REST (1): misunderstood and important, how and why

I want to take a break from declarative/curly posts and write a bit about REST. I hope this will help spawn some discussion among developers I work with and know.
I am on last pages of (the rightfully acclaimed) RESTful web services, by Richardson and Ruby book.  The reading felt a bit repetitive, it felt like hammering of ideas into my head, but they probably needed hammering.  Overall a great book, one that opened my mind to what REST is about (or should be about).

Somewhere half way through the book things started to click.   (I am trying to share some of these clicks in this and in a next few posts.)  I read quite a bit about REST before reaching for the book.  The first question is why did it take so long to click?  

I came out of this reading with a much better understanding of the HTTP protocol itself, so this must be a part of the learning curve.  But, I also blame my slowness on two things: first reason is the amount of confusion surrounding REST,  second reason is harder to explain:  REST is somewhat complex if you look at it as a set of implementation guidelines (the how),  it is also quite straightforward if you look at it as what it is for (the why).   It is hard to understand the how until you get the why.

Developers like to focus on the mechanics of things (the how).  Developers also have deadlines and work in a task-oriented environment where there maybe many whys but the goals of Uniform Interface, ease of future integration and benefits of semantic linking are probably not likely to be among the whys which pay their paychecks.  REST become such a buzzword that it ends up being used even if the why is not on the radar.  REST gets half-implemented but holds on to its full name. This fuels confusion.

REST Confused: 

The term REST is used a lot these days, I hear it at work,  I see a lot about it on the web.  Many developers are very interested in it and know a lot about it (way more than I do).  Last year's Uberconf REST offerings were very well attended.  At the same time, there is quite a bit of misunderstanding and confusion about what REST is.  (I was among the confused before reading this book.)  And I think that the industry-wide confusion on this topic will be winning.
The book authors decided to stay away from the REST term most of the time and have used the term ROA (Resource Oriented) to avoid the confusion.  (I stay with the terms REST and RESTful.)   
The book also introduced the therm REST-RPC Hybrid to describe many existing web services and to differentiate them from the true REST.  There is quite a bit of wit in this self-contradicting term.

It is just like Object Oriented
A somewhat forgotten old paper by Tim Rentsch (back in 1982) included a prophecy about what OO will become:  "Every manufacturer will promote his products as supporting it. Every manager will pay lip service to it. Every programmer will practice it (differently). And no one will know just what it is."

One more old OO related quote stays in my brain and does not want to leave:  I was at some sort of a conference and talking to vendors in the booth area.  One of them (he knew he is talking to a developer) was doing his spiel and said this: '... and we will be object oriented by the end of the next quarter'.   I am not trying to contribute ideas for a next Dilbert episode.  The scary thing is: that does not sound like something the marketing group would came up with on their own. I fear it came from the developers. 

I think REST became like OO from these last 2 quotes. Part of the problem is that, like OO, everyone wants to claim it, yet it is not so easy to implement (my next posts will talk more why it is hard) but the lip service is easy so the term is used a lot.

Here is a quote from the above book: "Both REST and web services have become buzzwords. They are chic and fashionable. These terms are artfully woven into PowerPoint presentations by people who have no real understanding of the subject".

Side NOTE:  a similarity between REST and OO:  both are often used as check boxes and both are really progress bars.

THE WHY:
I believe the fact that the term REST is misused and misunderstood causes lack of its true adoption.  We are missing out on some amazing opportunities.  REST could be a big step towards fulfilling a fantasy in which computer programs can surf the web the way humans do today.

Why REST is such a good idea?:  REST is the opposite of API.  APIs are all different, REST services are trying to be all the same.  

Let me quote the book:

"One alternative to explaining everything is to make your service look like other services.  If all services exposed the same representation formats, and mapped URIs to resources in the same way... well, we can't get rid of client programming altogether, but clients could work at higher level than HTTP."

I came up with this imprecise, sort-of definition of REST focusing on the why:  You wrote a very RESTful app if you have exposed your app functionality to the programmable web without API specification and without ambiguity.

I think the problem is that people think that REST is all about using GET, PUT, POST, DELETE (maybe OPTIONS, HEAD), some will know that POST as it stands today is better not used, but that is where it often ends.  We think of the mechanics and loose the big picture.  The net result is that we use the term REST a lot,  but what we develop typically has nothing to do with being easily accessible by the 'rest' of the programmable web.   

Example:

Say, you are developing a web application serving web pages and need to expose Ajax data from server to your Java Script library in a few places (just a GET request for simplicity). 

JSON is not very RESTful (point one: developers should be aware of the fact) but, if you decide on using JSON, still ask yourself these questions: 

• How cryptic is the URL, does it look anything like a location of a resource?  Any database IDs in it? (Would you expect external REST client to know your database IDs?)
• Can someone not very familiar with this project get any information from your JSON? 
• How much is this JSON specific to the current version of the UI:
• Is it just the fields that you need or a more meaningful representation? 
• Are you pre-formatting the data for your UI on the server?
• How is your error handling and status codes?  (say, you listing employees in a department and someone will issue a URL GET with invalid department: will your service just return 500 with the error message in plain text? - hey you know your JavaScript would never do that so why care about correct status codes or consistent content type?...).
• And finally how can another developer discover this URL?  Do you expect them to simply construct the URLs?

Lots of REST is simply common sense if you understand the why.

We are missing on big opportunities:

The first reason is so other developers can access/integrate easily with your application. 

The second reason is so that the future semantic web will find the data exposed by your app.

Such goal should be very important for businesses which want their products to be 'known' or educational institutions, organizations defining standards, many government institutions which want their data to be transparent and available to programs at large who may want to use it, study it, analyze it. 

These 2 reasons should create a big push for writing RESTful apps, but the reality is the exact opposite. Today businesses run hundreds of applications, these applications do not know how talk to each other.  Any integration is complex and expensive.  New applications are being developed (I participated in developing some of them) and REST is last thing on anyone's mind. 

The targets of semantic web today are not government, science, or even manufactured goods (although, Best Buy made recent news to the contrary), but tweeter, facebook, blogger, etc.  Who likes whom on facebook will benefit from all of this,  but is there any semantic web development for, say, endangered species?  This is not to criticize social web, kudos to them for innovating, this is to criticize everyone else who should be part of this and decides to stay behind.

Next post will have more mechanics:

I think a big part of the problem is that good REST is not so easy to implement. If it is hard and it is a buzzword people are bound to overuse it.  Buzzwords loose their meaning very fast, just like OO term did.

My next post will be about what is so hard about REST.