Stuff about programming, programming style, maintainability, testability. Dedicated to my coworkers and friends. Everyone is welcome to leave comments or disagree with me.

Friday, August 22, 2014

I don't like Hibernate/Grails part 5: auto-saving and auto-flushing

Probably the best testimony to GORM/Hibernate complexity is that the Grails project itself has hard time deciding what to do.  Interestingly, I recently found that the issue, I am about to present, is also discussed in The Definitive Guide to Grails 2  on page 217 (Automatic Session Flushing) and that what that book says is no longer true if you are using Grails 2.4.3!

Examples in all my previous posts have been verified on Grails 2.4.3 (currently latest) as well as on 2.3.3 (what I use at work).  This post shows GORM behavior that has changed since Grails 2.3.3.  I have tested both versions by creating new projects and accepting all default configurations.

Code Example:
Assume a very simple domain class which looks like this:
class Simple {
    String name
}


and a saved record with name 'simple1'. Here is a test case (asserts will fail using Grails 2.3.3 and will pass with 2.4.3):
    void testMysteriousSave() {
       
        Simple.withNewSession() {
            def simple1 = Simple.findByName('simple1')
            simple1.name = 'simple1b'
            Simple.findAll()
            simple1.discard()
        }
       
        Simple.withNewSession() {
            assert Simple.findByName('simple1')
            assert !Simple.findByName('simple1b')
        }
    }


(I have used findAll() just for simplicity but many other queries involving Simple class should cause a similar issue).  What is going on here?  GORM/Hibernate decides when to actually save objects (this is sometimes called write-behind approach but in this case it really is 'write-ahead of the developer').  Queries in GORM/Hibernate have a (yet one more) serious side-effect, they can persist objects stored on the session.  This behavior is controlled using FlushMode (see https://docs.jboss.org/hibernate/core/3.6/javadocs/org/hibernate/FlushMode.html).  GORM implementation must have recently changed how it uses FlushMode to avoid problems shown in the above code.

Side Note:  Here is how a similar code example is described in "The Definitive Guide to Grails 2" book: "You may consider the behavior of automatic flushing to be a little odd, but if you thing about it, it depends very much on your expectations. If the object weren't flushed to the database, then the change made to it on line 2 [refers to a line analogous to simple1.name = 'simple1b'] would not be reflected in the results [result of the finder on the following line]. That may not be what you're expecting either!".   Hmm, that appears to be simply not true.  GORM/Hibernate do not return latest data from the database,  that is why we have 'repeatable finder' problem discussed in post 2.   But if somehow this was all changed and Hibernate started working more like,  say, Active Record...  Why would that be confusing?  No, sorry, unexpected side-effects are never part of my expectations!

Side Note 2:  New project in Grails 2.4.3 will default this (new to me) setting in DataSource.groovy:
 hibernate {
    flush.mode = 'manual'
 }


changing this to 'auto' does not seem to cause any difference in how my test runs.  I believe the new behavior is a GORM code change not just a default project configuration change. (Please correct me if I am wrong.)

Why is (or was) this behavior dangerous:
There are many good reasons why I may want to keep validation logic outside of domain object, but this is/was a very risky thing to do.  Consider this simplistic controller pseudo-code (assume validate method sets errors on the object and returns false if validation fails):

def createNew() {
  DomainObject domainObject = DomainObject.findById(params.id)
  domainObject.parameters = params.data

  if(validate(domainObject)) {
     domainObject.save(flush:true, ...)
  } else {
     domainObject.discard()
  }
  return domainObject
}


Using Grails 2.3.3 that code was likely to save the object during validate call if validate method executed a 'wrong' query!  So here we have it again:  an example of code that works fine but will break if you add a 'finder' query.

My personal experience with this issue:
Non-transactional save/validation logic is only for brave Grails developers, I am not that brave.  Code that I work with uses transactional services to validate/create/update domain objects.  If such code fails any 'write-ahead of developer' saves are rolled back.  The problem, however, still exists.  Since the actual persisting can happen before validation logic completes, it is possible for Grails to try to save an object that, for example,  violates some DB enforced integrity checks. It is quite surprising if finder queries start throwing database update or insert errors!

How to find these errors:
Grails unit tests will not find them.  Integration tests and functional tests will find them.

Hindsight is 20/20?
The idea that side-effects are evil and that the ability to manage/isolate side-effects is what differentiates good programs from bad programs is not new, it predates Hibernate by decades.  Insert/update operations are serious side effects.  How can I manage these serious side-effects in my programs, if GORM/Hibernate hide these from me?  I find FlushMode a poor way to manage how objects are persisted to the database and the very idea of hiding these operations wrong in principle.

Future "I don't like" posts:
So far, I have taken a test-driven approach to these posts.  For each post I have created 'from scratch' a new Grails project with default configuration and wrote a test or series of tests to verify every problematic behavior I wanted to talk about.  This is not always easy to do.  There are some interesting side-effects that are hard to reproduce, possibly because of layers of configuration or something specific to a particular domain class.  For example, figuring out why grailsApplication.isDomainClass sometimes does not work (http://jira.grails.org/browse/GRAILS-11630) took some effort.

There are quite a few very interesting side effects for which I do not yet have a 'working' test case.  For example: GORM/Hibernate auto-saving is triggered by GORM/Hibernate thinking that the object is dirty.  I have seen a very surprising behavior around how dirty flag is set but, currently, I have no way to reproduce this behavior 'from scratch'.   This is a slow process and I am not sure I will succeed reproducing all issues. 
I want to give it couple of weeks before I write next installment.  At some point I may just write about things that I experienced, as opposed to things that I can demonstrate with a test case.
So this will not be my last post about Grails but most likely the next one will take couple of weeks to prepare.

Added 2014/08/25:
I have created this JIRA:  http://jira.grails.org/browse/GRAILS-11687
Current behavior of FlushMode is confusing.  This does not change the story in this post, but maybe will help clarify current behavior.

Friday, August 15, 2014

I don't like Grails/Hibernate, part 4. Hibernate proxy objects.

Hibernate uses proxy classes to implement its lazy loading of nested domain objects. As a result, instead of BankAccount class (defined in post 1)  I may sometimes get object with a class named something like 'BankAccount_$$_javassist_27'.

The idea behind the general concept of a proxy is that: use of proxy objects should be identical to the use of real object. This utopian wish is impossible to accomplish in practice, and Hibernate proxies are no exception.  Hibernate proxies can cause several problems, including
  • gotchas related to equals method implementation (not discussed), 
  • the famous LazyInitializationException (which I have promised not to discuss) 
  • casting problems caused by use of polymorphic associations (this is a well documented issue and I will stay away from it as well).
So, the next best thing proxy designers (in any framework) can do is to make sure that the use of their proxies is not very transparent to the developer.  Good examples to think about here are proxies that show up in remoting technologies like: the old CORBA, RMI, EJB or even SOAP based WebService (brrrr).  In these technologies, developer had to do some specific thing, like a JNDI namespace lookup, to get access to a proxy.  By contrast, in Grails/GORM/Hibernate it is very hard to maintain a mental map of which object is a real object and which is a proxy.  This causes very intermittent behavior.

Probably for that reason, proxies had a rough start in Grails.  The term 'proxy hell' was coined and some interesting bugs like this one:  http://jira.grails.org/browse/GRAILS-2570 have been reported.

Code Examples:
This code will get me a proxy object
  def id
  BankAccount.withNewSession {
     id = BankAccount.findByName('a1').id
  }
  def a1 = BankAccount.load(id)
  assert simple instanceof HibernateProxy


Note, that as usual with Hibernate, I can break the above code by adding an isolated query:
  BankAccount.findByName('a1')  //added this line

  def id
  BankAccount.withNewSession {
     id = BankAccount.findByName('a1').id
  }
  def a1 = BankAccount.load(id)
  assert simple instanceof HibernateProxy //now fails


You may find this last code example somewhat unrealistic, but I hope you agree with me that it explains the intermittent aspect of how proxies work (or why they often do not work).

How proxy code typically breaks:
Sooner or later, each Grails app will need to use domain object class to do various things with it like:
  • check if an object is a domain object or something else
  • introspect GORM properties on a domain object
  • find beans named after a domain object (for example, find BankAccountService if object is BankAccount)
And that is great, only, it is soooo easy to forget that, for example,    
    entity.getClass().shortName

may give me wrong class name! And if I do forget, my code will work, only, not always.

If you are very diligent in making sure that you always use the un-proxied classes only, then rest assured that someone else is not that diligent.  For example, here is a currently open Grails JIRA: http://jira.grails.org/browse/GRAILS-11630 and if you search source of, for example, various template libraries available out there you will find things like this code:
  grailsApplication.getArtefact(
       DomainClassArtefactHandler.TYPE, 
       element.getClass().name
 )

which, again, I am sure was tested and works ... only not always.

How to test for proxy problems:
This may sound like a broken record: Grails Unit Test will not find these.
Integration tests may find them but few developers code integration test separating these essential test elements into separate Hibernate sessions:
  • data setup (given)
  • test itself   (when)
  • test asserts (then)
(that lack of session isolation applies to Grails framework itself as shown by this open bug: http://jira.grails.org/browse/GRAILS-11644).  So Grails integration tests, as they are typically coded today, may not find most of proxy related issues. Functional tests are the best bet again.

Next topic: 
So far I have focused a class of Hibernate side-effects, where:
  BankAccount.findByName('a1') //(1)
  someOtherCode()              //(2)

adding isolated query (1) changes behavior of code (2).

I want to continue discussion about side-effects, but from a somewhat different angle. In post 1, I have promised not to talk about auto-saving, I will break that promise.  There are some very 'interesting' aspects of auto-saving worth examining,  for example, I sometimes get DB update operation errors from issuing a finder query.  I will try to examine a few of these interesting things.  Things that happen as a consequence of Hibernate design decision to make a major side-effect, like SQL update or insert operation, implicit and invisible.

Monday, August 11, 2014

I don't like Grails/Hibernate part 3. DuplicateKeyException: Catch it if you can.

This post follows a pattern I used so far:  it documents a case where adding an isolated query can break Grails/Hibernate code. 

I often think that a measure of well designed library is how in how it handles exceptional cases.  Hibernate does not 'exception' well, but Hibernate behind Spring Framework Templates, and then behind GORM can be really puzzling.  In this post, I will examine one of such puzzling cases.

Continuing my previous post, which described a very scary 'repeatable finder' concurrency issue. There are no good solutions to that problem and identifying affected application areas seems close to impossible.  However, repeatable finder problem can be partially solved by using shorter, more atomic Hibernate sessions.

Call for short Hibernate Sessions:
By design, Hibernate sessions are implicit in Grails and it is the framework responsibility to manage hibernate session life cycle when processing HTTP requests. Taking over that role does not seem to be a good idea.  With that said, since there are no good solutions to the 'repeatable finder' problem, bad solution maybe still the best I got!  Also, there are other, better acknowledged, reasons why I may need to control hibernate sessions, such as performance, long running scheduled jobs, Grails integration tests.

I find the whole issue a bit ironic.  Web developers have been, by now, conditioned to minimize the use of HTTP session.  REST wants to ban it all together.  Yet, Hibernate design is to maximize the use of hibernate sessions. Both are shared application state, if one is bad so should be the other!   Is this the idea: that shorter lived evil is less evil so it is OK to use it for everything?  If that is so, here you have it, one more argument for making hibernate session shorter.

There is an API to interact with hibernate sessions.  I can use sessionFactory bean directly to flush/close/create sessions or I can use withNewSession method available on any GORM domain object.  

Unfortunately, dealing with more than one Hibernate session exposes me to a bunch of Hibernate/Spring exceptions that would be otherwise unknown to me as a Grails developer:  HibernateSystemException, NonUniqueObjectException (hibernate), and DuplicateKeyException (spring) are among them. I will focus on the last 2.

NonUniqueObjectException (hibernate), and its twin DuplicateKeyException (spring):
In my experience so far, they seem to be linked to each other (DuplicateKeyException wraps hibernate NonUniqueObjectException).  I had hard time finding good documentation about these two, documentation that is relevant to how Grails works.  Hibernate JavaDoc for NonUniqueObjectException gives me only this: 

"This exception is thrown when an operation would break session-scoped identity. This occurs if the user tries to associate two different instances of the same Java class with a particular identifier, in the scope of a single Session." (http://docs.jboss.org/hibernate/core/3.6/javadocs/org/hibernate/NonUniqueObjectException.html)

This is not something I, as a Grails developer, want to identify with.  Instead, I would prefer the framework to enforce that objects returned using query in one session are not used in other session. But that is not exactly what the error indicates or not what it is.
Please note that Hibernate is not very logical here either:  it is not exactly that the user always  'associates' instances with the session.   They can get associated sometimes in ways that would surprise most of the users! (I may need to post about it too.)  Hibernate does not provide any public API to query for what is associated with the session.  It considers this 'private' information.  Well, if it is so private that I can't even query for it, why am I seeing it then in the exception?
DuplicateKeyException documentation seems simply incorrect for the context in which I am seeing this error:

"Exception thrown when an attempt to insert or update data results in violation of an primary key or unique constraint. Note that this is not necessarily a purely relational concept; unique primary keys are required by most database types." (http://docs.spring.io/spring/docs/3.0.x/api/org/springframework/dao/DuplicateKeyException.html)

Fortunately, the message I typically get is more descriptive: "a different object with the same identifier value was already associated with the session".   So both documentation and exception design seems to be a mess here, but the real mess is still ahead of us.

Code Examples:
As a Grails developer, you may find it surprising that this code even works:
        def ac1 = BankAccount.findByName('a1')

        BankAccount.withNewSession { session2->
            ac1.name = 'a1b'
            ac1.save(flush:true, failOnError: true)
        }

it is better to see the problem if I make the code more Hibernate explicit (which still works just fine):
        def ac1 = BankAccount.findByName('a1')

        BankAccount.withNewSession { session2->
            ac1.name = 'a1b'
            session2.saveOrUpdate(ac1)
        }

If ac1 is associated with my first session and not session2, why session2 allows me to save it?  Would it be not more logical if this code threw an exception with something like 'not associated with session'? This may make sense for more general case that Hibernate tries to accommodate, but it makes Grails behave inconsistently. 
Now, I can break it by adding a finder:        
        def ac1 = BankAccount.findByName('a1')

        BankAccount.withNewSession {
            BankAccount.findByName('a1') //added this line
            ac1.name = 'a1b'
            println shouldFail(org.springframework.dao.DuplicateKeyException) {
                ac1.save(flush: true, failOnError: true)
            }
        }

or to be more Hibernate explicit:
        def ac1 = BankAccount.findByName('a1')
        def id = ac1.id

        BankAccount.withNewSession { session->
            session.get(BankAccount, id)
            ac1.name = 'a1b'
            println shouldFail(org.hibernate.NonUniqueObjectException) {
                session.update(ac1)
            }
        }

And the names find/get sound so innocent ...  Imagine running a diff, comparing what changed from last stable source code version to figure out what caused the problem: and finding only extra finder methods!

Again, one sane way to think of this issue is that I am using ac1 associated with session1 on a wrong session (session2) and that is wrong.  But if that is the case WHY does my first example work!  

SIDE NOTE:  In my experience, this is not the only way to get into DuplicateKeyException trouble and I have not figured out all Grails code triggers for it. In most cases, I was able to solve the problem by 'bringing' some domain object into the current session. So the mechanics of the problem seem to be always on some level similar to what I have described. 

How to test for these?
Grails unit test coverage will be useless for finding DuplicateKeyException/NonUniqueObjectException.
Both integration and Functional tests are capable of finding this issue.

Why Grails has done it this way?
From what I know, GORM tries to be a thin Groovy layer around Spring Hibernate Templates. In addition, Hibernate does not expose any public API to query what domain objects have been attached to the session so GORM would have to remeber that.  One solution could be for GORM to store 'owning' session on each domain class created by Grails and use it to provide more meaningful and consistent exception if client code tries to use it in a context of another session.

Refrences:

Summarizing examples shown so far:
In its ORM pursuit, Hibernate has lost something much more fundamental and infinity more important than purist ORM thing can possibly be.  Ability to manage unwanted side effects has been lost and, as we have seen in a couple of examples already.  In code like this:

   BankAccount.findByName('a1') //(1)
   someOtherCode()              //(2)

(1) can change behavior of (2), in most extreme case it can break it. 

As a result, ability to decouple application logic is largely lost if I use Grails/Hibernate stack.  I consider this a major Hibernate design flaw, but because GORM Domain Objects are likely to be used extensively in Grails apps, Grails applications are more impacted by it.  

Also note that problems like this maybe very hard to troubleshoot.  Even if I somehow manage to have a mental image of every finder, every eagerly loaded association, Grails/Hibernate can (and will) put objects in the cache that will surprise anybody. (I may write about it too).

In my next post I will examine the same pattern (adding isolated query breaks Grails code) in a context of Hibernate proxies and talk about another related Grails bug.