Software has two ingredients: opinions and logic (=programming). The second ingredient is rare and is typically replaced by the first.
I blog about code correctness, maintainability, testability, and functional programming.
This blog does not represent views or opinions of my employer.

Saturday, October 25, 2014

I don't like Hibernate/Grails part 11. Final thoughts.

This series was born out of my frustration with Grails. But, instead of making it a comprehensive criticism of the framework, I have decided to focus on a few GORM and Hibernate issues. I had several reasons to do that.

Why GORM/Hibernate focus?
There is quite a few blogs which basically say: Grails is very buggy and then provide few or no details. There are also many blogs saying Grails is great and then provide equal amount of fluff to support their claim. All of this becomes very subjective.

(Section Edited for clarity, Oct 30, 2014)
It is not that hard to demonstrate that this is a very buggy environment. It has been founded on Groovy, and, in my experience, Groovy is and always was is a very buggy language.  Here is one curious example (tested with Groovy 2.3.6, other versions I checked behave the same way):
  1 as Long == 1 as Integer //true (note, false in Java)
  1 as Integer == 1 as Long //true (note, false in Java)
  [(1 as Integer)].contains((1 as Long))  //false (inconsistent with equals!)
  (1 as Long) in [(1 as Integer)] //false

That is some scary stuff, right?  I think it is scary. This one is not very fixable,  but most Groovy bugs will eventually get fixed.  Hibernate bugs will be with us forever. This series was about bugs other people call functionality and sophisticated design.

Designing a good web framework is not easy.  I think the key is: the framework must be intuitive. It should do what developers expect to happen. That is one more reason for my focusing on GORM/Hibernate. It is hard to claim that these 2 are intuitive, but a similar claim becomes much more subjective in other parts of Grails.
Here is one example of a non-intuitive behavior: How does controller forwarding work. The intuitive behavior would be that forwarded method executes in a separate thread. It does not, it executes as part of the calling method. Instead of wasting time on disagreeing that this is a bad design, do this experiment: Create a controller with methods ‘a’ and ‘b’ and have 'a' forward to 'b'.  Add a filter which simply prints action name in ‘before’ and ‘afterView’.  Here is what you will get with Grails 2.4.3:
    before a,  before  b,  afterView b,  afterView b 
(both afterView print the same action name!).  Would it not be nicer if we got:
    before a,  afterView a,  before b,   afterView b      
Confusing design + mutating state == bugs. But how can I argue that this is not just an innocent overlook on the part of Grails/Spring framework?

One of the most irritating aspects of Grails is that everything is so intermittent, and that is by design. The fail-fast philosophy is totally foreign to this framework. For example, if calling object.save() no longer always saves the object (yes you are reading it right, see GRAILS-11797GRAILS-11536) then would you not want object.save() to fail if the object is not going to be ever saved?  Again, focusing on GROM/Hibernate simplified my job of demonstrating examples with a very intermittent behavior.
The uncanny ability to exacute bad code in Grails goes way beyond what I was able to demonstrate. This is the very scary: How did that ever work before? thing. I suspect something is wrong with Grails/Groovy compilation and a bad code sometimes magically works until some totally not relevant change exposes the problem. I cannot justify this claim. The only think I have is anecdotal evidence.

I have very strong opinions about what are the main causes of OOP bugs. That does not mean you would have agreed with me. Without good examples, this blog would have been labeled as a one more guy that thinks that 'FP is the new silver bullet'. Focusing on GORM allowed me to pinpoint the problems in a way that is hard to dispute. (And yet, I still got the label.)

Is it all about the cache?
All of the GORM problems listed in this series can be attributed in some way to Hibernate session/1st level cache.  You can argue that having cache is beneficial and that some problems are unavoidable with any cache implementation.
Ideally, caching should behave as if it was not there: application using a cache should work exactly the same way if the cache was removed. However, synchronization of ORM cache and DB state is a very hard problem and achieving ‘opaque’ implementation may be hard or even impossible.

Dear GORM/Hibernate:  If you can’t implement cache which behaves ‘like it is not there’ then don’t design your API like the cache ‘is not there’.  Make the cache very explicit and optional (Identity Map?). Invalidate cache immediately, or at least, provide a way for the application to learn as soon as you know that cached data is stale. Design invalidating (you like to call it session.clear()) your cache in a way that does not make half of objects used by the application useless.  Remember, you are just a cache, the data is still there!  The data is what is important, not you.  If you want to call yourself a cache, stop being so bossy!  :)

More criticism of Hibernate
I must point out that I am not the only Hibernate hater. Here are some examples:
http://www.slideshare.net/alimenkou/why-do-i-hate-hibernate
http://mentablog.soliveirajr.com/2012/11/hibernate-is-more-complex-than-the-problem-it-tries-to-solve/
http://brian.pontarelli.com/2007/04/03/hibernate-pitfalls-part-2/

In my blog, I have just scratched the surface.  Some examples of 'bug generating' design that could use more discussion include: defaulted and recommend 'flush:false' in save() (maybe a moot point now), or what happens when Hibernate session closes suddenly and unexpectedly (like, when transactional code fails).
I need to stop somewhere and this post seems a good place and time to stop.

Sneaking-in FP and other things that interest me
I decided that complaining about Grails not working right is much less powerful than pointing out why it does not work right.  I think analyzing and criticizing bad programs is a great way to advance programming skills. With each installment I tried to sneak-in some concept that explained why bad is bad: properties, shared state/side-effects, fail-fast, unit testability, even a bit of combinators. These are all common sense things that explain design flaws.

One thing I could not fit into my posts was types.  I decided that this will be too foreign concept in the context of Java and Groovy. Types are very powerful and I regret not finding a good place for them in this series.

Is FP the ‘new’ silver bullet?
I was asked this question and it makes sense that I try to answer it.
FP is not new, FP predates OOP,  Haskell is older than Java, combinatory logic is older than Turing machines, lambda calculus is about the same age.  The silver bullet is and probably always was the ability to logically reason on the code.

In this series, I tried to emphasize the importance of logic.  Programs should be logically simple and ‘mappable’ to logic. Programming and Logic are very related on a theoretical level (google: Curry-Howard).  This 3 are called the trinity of CS:  Type Theory, Category Theory and Proof Theory (google: Curry-Howard-Lambek).

Programs I write using Groovy and Grails may look straightforward and shorter than Java and Spring but from the point of view of logic these are still just spaghetti threads of programming instructions. Add to it a total disregard for side-effects and this exercise becomes equivalent to building a house of cards on a foundation that is shaking.

Quiz Question:  Recalling Logic 101, here is a logical ‘formula’:
    (a^b)=>c   ⇔   a=>(b=>c)
Do you know/can you figure out what that corresponds to in programming?  Answer at the end of this post.

I used to love Grails
I started this blog site with a series about 'Imperative curlies'.  My idea at that time was that I can count the number of curly braces ({}) in my code and use that number as a measure of how good my code is. The fewer 'curlies', the better the code.  The idea was to break away from coding and thinking using imperative sequences of instructions (for loops, if statements all use 'curlies').  I remember it worked very well for me. If you look at these old posts, you will see that there was a time when I really liked Grails.

Is all OOP bad?
I think that is a complex question.  Good OOP is about things like decoupling, separation of concerns, eliminating shared state, meaningful polymorphism, etc. These things may achieve some of the same goals FP is fighting for. The concept of a shared session state (Hibernate session) is not very OO.  Hibernate StatelessSession interface which does not extend Session is not a great example of OO polymorphism. Ability to decouple is mostly gone due to Hibernate non-localized side-effects. Hibernate is simply not a good OOP.

What any OOP will always lack is this: a clear and simple correspondence to logic. This is what makes FP unique.

Parting thought:
Answer to the Quiz Question:  It is currying. To see it, compare these 2 lines:
   (a^b)=>c              ⇔    a=>(b=>c)
   (a,b)->c              ⇔    a->(b->c)
   (2 argument function)       (function returning a function) 
First line is the logical formula. I have changed arrow-like symbols '=>' to look slightly different '->'. I have replaced '^' with ',' and ended up in FP!  This process is a mini-Category Theory in action. Cool, is it not?

There is no helping it, Groovy and Grails are OOP not FP. It is still important to be able to think outside of that box. Otherwise we will start convincing ourselves of things like ‘static definitions are always bad’, ‘unit testing is not about finding bugs’, ‘using refresh() resolves stale object problems in Hibernate session’,  or some other nonsense.

Thinking in C++, Thinking in Java: it is worth trying to stop it, even if you program in these languages.

Grails is and will be a very popular and buggy framework. We can only blame ourselves for that. Writing this series was a big effort for me. My biggest hope is that it made some of my readers stop for a moment with a 'hmm'.

The End (for now).

(I ended up republishing this blog due to some weird formatting issues -  if I created a chaos in your RRS/Atom feed - sorry!)

8 comments:

  1. I am not a groovy expert but I am a little bit surprised on how you generalize "Groovy is and always was is a very buggy language" and then explain this with an example which (IMHO) has nothing to do with groovy:

    the collections GDK has no 'contains()' method (http://groovy.codehaus.org/groovy-jdk/java/util/Collection.html) . So this method is taken from the JDK (http://docs.oracle.com/javase/7/docs/api/java/util/AbstractCollection.html). It might be more of a java problem and since groovy is build on top of java...

    But still, I wouldn't count this as a buggy feature. The method name might suggests another behaviour, but is this a bug?

    I use groovy since some years now and didn't (as far as I remember) run into bugs. Maybe "unexpected" or "not intuitive" behaviour, but no bugs ;-)

    ReplyDelete
    Replies
    1. "Groovy is and always was is a very buggy language"
      Well, that was my experience. Some examples, if I remember correctly: I had various problems with @Delegate annotation/transformation, 'minus' intermittently not working in lists (but working fine in Sets), etc.

      I agree that the example I gave maybe not the best one. It is an issue introduced by Groovy, not Java but it is something that is not easy fixable - more of a design issue.

      I will correct the text in this post to explain my position on this better.

      Delete
  2. Very good read. thanks for sharing your detailed impressions and your experience on this topic.

    About 3 years ago we decided to change our stack from a UML based AndroMDA/Maven/Spring/Hibernate/JSF way to Grails.
    The productivity boost was enormous. And yes there are some really really hard edges when working with grails.
    But right now we hardly see an alternative on the JVM.

    In fact there are two thing i would like to talk to you:

    1.Hibernate / GORM

    Yes. We been through this too. There are many situations where one can get into serious trouble.
    Just a few weeks ago we had the situation that a custom validator failed after binding.
    The domain object was updated in the hibernate session but did not get persisted…
    But as i said - there is no alternative. So our approach is 2 way:

    Either it`s a trivial 1 Domain CRUD controller - which can easily handle everything w/o a service.
    Then we got some abstractions and work with (pseudo) 'read-only‘ objects.
    Looks like:

    def edit(){
    withDomainFooOrNewInstance{ d ->
    whenFormPost{
    d = domainSave{
    domainSet('name','age') << d
    }
    }
    renderInto("edit") << [d:d]
    }
    }

    As soon as we take care of more than 1 domain or take use of command objects we prefer a service.

    So our way to tame hibernate/GORM and its statefulness is:
    - read only as possible
    - explicit save
    - no finders
    - named queries

    But as i told you: we used to write applications with JSF where even bugs have a lifecycle and a state.
    So things were already worse... =)

    2.FP

    'There is no helping it, Groovy and Grails are OOP not FP.'

    Again. Yes. Absolutely. Groovy is no functional language. It can try to act like one.
    But Java`s type system just prevents any higher kinded things. Plus plain Groovy / Java code is full of side effects,
    due to underlaying frameworks, or langugae shortcomings. Most of those APIs were written based on the principles of OOP.
    (You pass 3 Objects to a method and all got changed inside - surprise)
    So everytime you try to wrap parts in a functional way you have to bend things.
    Good for us that we also take use of Javascript and Swift, where writing functional parts is a little bit more easy. =)

    So what to do?

    We aren`t attached to grails.
    We need high productivity.
    We need great extensibility.
    And of course robustness.

    Have you considered exploring 'real' functional web framework like yesod?
    Have you plans looking more into scala / play?

    Again - thanks for sharing all your experiences.
    Cheers - Elmar

    ReplyDelete
    Replies
    1. (1) Hibernate/GORM: I think, keeping things as simple and straightforward as possible is a good thing. Moving forward, I would want to consider using less GORM and more direct SQL. I am no longer sure if ORM gives me that much, definitely using GORM has influenced my view on this.

      (2) FP: I wish it was a simple choice. I think going functional or selecting language/technology stack has to be a team decision, often is a company-wide decision. That makes things hard. Scala is considered a steep learning curve, it that is true, Haskell is advanced rock climbing ;) Changing technology is expensive process.

      I have not 'played' with Play. I have no clue how good it is. I think it has a chance of being a good alternative to Grails because it can be used with plain Java. That would allow for slower introduction of Scala. The fact that Play has recently incorporated Hibernate is a big surprise and a turnoff for me.
      Still, if changing technology was an option for me, I would want to dedicate some time and have a deeper look at Play.

      I have not played with Yesod either, or any other Haskell web frameworks. That is in part because I do not see it as a viable option in my current workplace.
      I think Haskell is not something I would be able to sell as a development language at my workplace.
      I know some developers who went through a successful progression in their work: Java -> Scala -> Haskell, but Java/Groovy -> Haskell is probably not something that can realistically happen unless all developers want to do it.
      More incremental approach allows people to see benefits of moving more toward FP. With Java/Groovy mindset Haskell is just to abstract to be enticing.

      There are some web frameworks built for Clojure, but I do not know much about them. There is also .Net and people like Erik Meijer have left their fingerprints on it. It is not functional but probably much more solid than Grails ever will be.

      There is also a possibility of building a custom technology stack. I prefer out of the box solution and I would not want to try selling that idea to my coworkers. Still lots of people do that, I have seen in done with Haskell, read about doing this using Clojure and Ring.

      Thanks for your reply. I do not have a good answer for what is the best alternative for a web framework in the JVM world. I wish I did. For sure Scala seems to me a more solid language foundation and is something worth investigating as a direction.

      Delete
  3. The only problem i see with scala/play (which i don't know if it still exists) is the binary incombatibility.
    But as i have no experiences maintaining a scala project i don't know if it's a real problem.

    Moving back to a more SQL like connection may be an option. I think slick / scala has some neat features / syntax.
    My favorite scenario - everything is read only - get an "IOMonad" and perform explicit save.

    I dont know if we will move on to scala.
    There has been a lot of discussion about the language and its development lately.
    So i think we will stay with grails for this year.
    After having more experience with swift, we may have a better opinion towards scala next year.

    we will see.
    btw - you have any twitter account so that its easier to stay in touch?

    ReplyDelete
    Replies
    1. Sorry I am a bit old century as far as social media. I barely do facebook, no twitter. Google +?
      Ideally I would like to have something like SQLMonad that is more specific than IO.
      Good luck! My group made a big investment in Grails and most likely we will be staying with Grails for some time.

      Delete
  4. Thanks for the effort in documenting these pitfalls.

    I have recently read your comments on GORM/Hibernate and mostly agree, for your style of complex application.

    There is another type of complex application that I think of as Enterprise applications. These tend to be wide, not deep in complexity. They have a lot of domain objects that interact in somewhat complex services. These services tend to read and use these domain objects and not change them. Most of the time, the read is done via the get(id). This reduces the chance of the bad session behavior you describe.

    In my type of complex app, the true primary keys (as known by the user) don't change. When you create an order 'M1001', it stays 'M1001'. My users rarely change it. I can see how using finders on a key that changes can cause problems.

    So far I have only run into one type of bad session behavior in one scenario: trying to update records in a beforeValidation() method. I solved that using a 'withNewSession' closure around the update logic.

    I have looked far and wide to find a framework that reduces boilerplate code and makes the business logic as clear and simple as Groovy/Grails does. So far, nothing comes close. Some will generate a pile of code for you, but you still have to maintain all of that generated code.

    I have noticed with Grails, when you deviate from the expected approach, Grails will cause problems someday.

    ReplyDelete