I don't like Grails/Hibernate part 3. DuplicateKeyException: Catch it if you can.

This post follows a pattern I used so far:  it documents a case where adding an isolated query can break Grails/Hibernate code. 

I often think that a measure of well designed library is how in how it handles exceptional cases.  Hibernate does not 'exception' well, but Hibernate behind Spring Framework Templates, and then behind GORM can be really puzzling.  In this post, I will examine one of such puzzling cases.

Continuing my previous post, which described a very scary 'repeatable finder' concurrency issue. There are no good solutions to that problem and identifying affected application areas seems close to impossible.  However, repeatable finder problem can be partially solved by using shorter, more atomic Hibernate sessions.

Call for short Hibernate Sessions:

By design, Hibernate sessions are implicit in Grails and it is the framework responsibility to manage hibernate session life cycle when processing HTTP requests. Taking over that role does not seem to be a good idea.  With that said, since there are no good solutions to the 'repeatable finder' problem, bad solution maybe still the best I got!  Also, there are other, better acknowledged, reasons why I may need to control hibernate sessions, such as performance, long running scheduled jobs, Grails integration tests.

I find the whole issue a bit ironic.  Web developers have been, by now, conditioned to minimize the use of HTTP session.  REST wants to ban it all together.  Yet, Hibernate design is to maximize the use of hibernate sessions. Both are shared application state, if one is bad so should be the other!   Is this the idea: that shorter lived evil is less evil so it is OK to use it for everything?  If that is so, here you have it, one more argument for making hibernate session shorter.

There is an API to interact with hibernate sessions.  I can use sessionFactory bean directly to flush/close/create sessions or I can use withNewSession method available on any GORM domain object.  

Unfortunately, dealing with more than one Hibernate session exposes me to a bunch of Hibernate/Spring exceptions that would be otherwise unknown to me as a Grails developer:  HibernateSystemException, NonUniqueObjectException (hibernate), and DuplicateKeyException (spring) are among them. I will focus on the last 2.

NonUniqueObjectException (hibernate), and its twin DuplicateKeyException (spring):

In my experience so far, they seem to be linked to each other (DuplicateKeyException wraps hibernate NonUniqueObjectException).  I had hard time finding good documentation about these two, documentation that is relevant to how Grails works.  Hibernate JavaDoc for NonUniqueObjectException gives me only this: 

"This exception is thrown when an operation would break session-scoped identity. This occurs if the user tries to associate two different instances of the same Java class with a particular identifier, in the scope of a single Session." (http://docs.jboss.org/hibernate/core/3.6/javadocs/org/hibernate/NonUniqueObjectException.html)

This is not something I, as a Grails developer, want to identify with.  Instead, I would prefer the framework to enforce that objects returned using query in one session are not used in other session. But that is not exactly what the error indicates or not what it is.
Please note that Hibernate is not very logical here either:  it is not exactly that the user always  'associates' instances with the session.   They can get associated sometimes in ways that would surprise most of the users! (I may need to post about it too.)  Hibernate does not provide any public API to query for what is associated with the session.  It considers this 'private' information.  Well, if it is so private that I can't even query for it, why am I seeing it then in the exception?

DuplicateKeyException documentation seems simply incorrect for the context in which I am seeing this error:

"Exception thrown when an attempt to insert or update data results in violation of an primary key or unique constraint. Note that this is not necessarily a purely relational concept; unique primary keys are required by most database types." (http://docs.spring.io/spring/docs/3.0.x/api/org/springframework/dao/DuplicateKeyException.html)

Fortunately, the message I typically get is more descriptive: "a different object with the same identifier value was already associated with the session".   So both documentation and exception design seems to be a mess here, but the real mess is still ahead of us.

Code Examples:

As a Grails developer, you may find it surprising that this code even works:

        def ac1 = BankAccount.findByName('a1')

        BankAccount.withNewSession { session2->

            ac1.name = 'a1b'

            ac1.save(flush:true, failOnError: true)

        }

it is better to see the problem if I make the code more Hibernate explicit (which still works just fine):

        def ac1 = BankAccount.findByName('a1')

        BankAccount.withNewSession { session2->

            ac1.name = 'a1b'

            session2.saveOrUpdate(ac1)

        }

If ac1 is associated with my first session and not session2, why session2 allows me to save it?  Would it be not more logical if this code threw an exception with something like 'not associated with session'? This may make sense for more general case that Hibernate tries to accommodate, but it makes Grails behave inconsistently. 

Now, I can break it by adding a finder:        

        def ac1 = BankAccount.findByName('a1')

        BankAccount.withNewSession {

            BankAccount.findByName('a1') //added this line

            ac1.name = 'a1b'

            println shouldFail(org.springframework.dao.DuplicateKeyException) {

                ac1.save(flush: true, failOnError: true)

            }

        }

or to be more Hibernate explicit:

        def ac1 = BankAccount.findByName('a1')

        def id = ac1.id

        BankAccount.withNewSession { session->

            session.get(BankAccount, id)

            ac1.name = 'a1b'

            println shouldFail(org.hibernate.NonUniqueObjectException) {

                session.update(ac1)

            }

        }

And the names find/get sound so innocent ...  Imagine running a diff, comparing what changed from last stable source code version to figure out what caused the problem: and finding only extra finder methods!

Again, one sane way to think of this issue is that I am using ac1 associated with session1 on a wrong session (session2) and that is wrong.  But if that is the case WHY does my first example work!  

SIDE NOTE:  In my experience, this is not the only way to get into DuplicateKeyException trouble and I have not figured out all Grails code triggers for it. In most cases, I was able to solve the problem by 'bringing' some domain object into the current session. So the mechanics of the problem seem to be always on some level similar to what I have described. 

How to test for these?

Grails unit test coverage will be useless for finding DuplicateKeyException/NonUniqueObjectException (EDITED: use of HibernateTestMixin may change that).

Both integration and Functional tests are capable of finding this issue.

Why Grails has done it this way?

From what I know, GORM tries to be a thin Groovy layer around Spring Hibernate Templates. In addition, Hibernate does not expose any public API to query what domain objects have been attached to the session so GORM would have to remeber that.  One solution could be for GORM to store 'owning' session on each domain class created by Grails and use it to provide more meaningful and consistent exception if client code tries to use it in a context of another session.

Refrences:

Grails JIRA: https://jira.grails.org/browse/GRAILS-11652

Summarizing examples shown so far:

In its ORM pursuit, Hibernate has lost something much more fundamental and infinity more important than purist ORM thing can possibly be.  Ability to manage unwanted side effects has been lost and, as we have seen in a couple of examples already.  In code like this:

   BankAccount.findByName('a1') //(1)

   someOtherCode()              //(2)

(1) can change behavior of (2), in most extreme case it can break it. 

As a result, ability to decouple application logic is largely lost if I use Grails/Hibernate stack.  I consider this a major Hibernate design flaw, but because GORM Domain Objects are likely to be used extensively in Grails apps, Grails applications are more impacted by it.  

Also note that problems like this maybe very hard to troubleshoot.  Even if I somehow manage to have a mental image of every finder, every eagerly loaded association, Grails/Hibernate can (and will) put objects in the cache that will surprise anybody. (I may write about it too).

In my next post I will examine the same pattern (adding isolated query breaks Grails code) in a context of Hibernate proxies and talk about another related Grails bug.