I don't like Hibernate/Grails part 5: auto-saving and auto-flushing

Probably the best testimony to GORM/Hibernate complexity is that the Grails project itself has hard time deciding what to do.  Interestingly, I recently found that the issue, I am about to present, is also discussed in The Definitive Guide to Grails 2  on page 217 (Automatic Session Flushing) and that what that book says is no longer true if you are using Grails 2.4.3!

Examples in all my previous posts have been verified on Grails 2.4.3 (currently latest) as well as on 2.3.3 (what I use at work).  This post shows GORM behavior that has changed since Grails 2.3.3.  I have tested both versions by creating new projects and accepting all default configurations.

Code Example:
Assume a very simple domain class which looks like this:
class Simple {
    String name
}

and a saved record with name 'simple1'. Here is a test case (asserts will fail using Grails 2.3.3 and will pass with 2.4.3):
    void testMysteriousSave() {
       
        Simple.withNewSession() {
            def simple1 = Simple.findByName('simple1')
            simple1.name = 'simple1b'
            Simple.findAll()
            simple1.discard()
        }
       
        Simple.withNewSession() {
            assert Simple.findByName('simple1')
            assert !Simple.findByName('simple1b')
        }
    }

(I have used findAll() just for simplicity but many other queries involving Simple class should cause a similar issue).  What is going on here?  GORM/Hibernate decides when to actually save objects (this is sometimes called write-behind approach but in this case it really is 'write-ahead of the developer').  Queries in GORM/Hibernate have a (yet one more) serious side-effect, they can persist objects stored on the session.  This behavior is controlled using FlushMode (see https://docs.jboss.org/hibernate/core/3.6/javadocs/org/hibernate/FlushMode.html).  GORM implementation must have recently changed how it uses FlushMode to avoid problems shown in the above code.

Side Note:  Here is how a similar code example is described in "The Definitive Guide to Grails 2" book: "You may consider the behavior of automatic flushing to be a little odd, but if you thing about it, it depends very much on your expectations. If the object weren't flushed to the database, then the change made to it on line 2 [refers to a line analogous to simple1.name = 'simple1b'] would not be reflected in the results [result of the finder on the following line]. That may not be what you're expecting either!".   Hmm, that appears to be simply not true.  GORM/Hibernate do not return latest data from the database,  that is why we have 'repeatable finder' problem discussed in post 2.   But if somehow this was all changed and Hibernate started working more like,  say, Active Record...  Why would that be confusing?  No, sorry, unexpected side-effects are never part of my expectations!

Side Note 2:  New project in Grails 2.4.3 will default this (new to me) setting in DataSource.groovy:
 hibernate {
    flush.mode = 'manual'
 }

changing this to 'auto' does not seem to cause any difference in how my test runs.  I believe the new behavior is a GORM code change not just a default project configuration change. (Please correct me if I am wrong.)

Why is (or was) this behavior dangerous:
There are many good reasons why I may want to keep validation logic outside of domain object, but this is/was a very risky thing to do.  Consider this simplistic controller pseudo-code (assume validate method sets errors on the object and returns false if validation fails):

def createNew() {
  DomainObject domainObject = DomainObject.findById(params.id)
  domainObject.parameters = params.data

  if(validate(domainObject)) {
     domainObject.save(flush:true, ...)
  } else {
     domainObject.discard()
  }
  return domainObject
}

Using Grails 2.3.3 that code was likely to save the object during validate call if validate method executed a 'wrong' query!  So here we have it again:  an example of code that works fine but will break if you add a 'finder' query.

My personal experience with this issue:
Non-transactional save/validation logic is only for brave Grails developers, I am not that brave.  Code that I work with uses transactional services to validate/create/update domain objects.  If such code fails any 'write-ahead of developer' saves are rolled back.  The problem, however, still exists.  Since the actual persisting can happen before validation logic completes, it is possible for Grails to try to save an object that, for example,  violates some DB enforced integrity checks. It is quite surprising if finder queries start throwing database update or insert errors!

How to find these errors:
Grails unit tests will not find them (EDITED: use of HibernateTestMixin may change that. I have not tried it).  Integration tests and functional tests will find them.

Hindsight is 20/20?
The idea that side-effects are evil and that the ability to manage/isolate side-effects is what differentiates good programs from bad programs is not new, it predates Hibernate by decades.  Insert/update operations are serious side effects.  How can I manage these serious side-effects in my programs, if GORM/Hibernate hide these from me?  I find FlushMode a poor way to manage how objects are persisted to the database and the very idea of hiding these operations wrong in principle.

Future "I don't like" posts:
So far, I have taken a test-driven approach to these posts.  For each post I have created 'from scratch' a new Grails project with default configuration and wrote a test or series of tests to verify every problematic behavior I wanted to talk about.  This is not always easy to do.  There are some interesting side-effects that are hard to reproduce, possibly because of layers of configuration or something specific to a particular domain class.  For example, figuring out why grailsApplication.isDomainClass sometimes does not work (http://jira.grails.org/browse/GRAILS-11630) took some effort.

There are quite a few very interesting side effects for which I do not yet have a 'working' test case.  For example: GORM/Hibernate auto-saving is triggered by GORM/Hibernate thinking that the object is dirty.  I have seen a very surprising behavior around how dirty flag is set but, currently, I have no way to reproduce this behavior 'from scratch'.   This is a slow process and I am not sure I will succeed reproducing all issues. 
I want to give it couple of weeks before I write next installment.  At some point I may just write about things that I experienced, as opposed to things that I can demonstrate with a test case.
So this will not be my last post about Grails but most likely the next one will take couple of weeks to prepare.

Added 2014/08/25:
I have created this JIRA:  http://jira.grails.org/browse/GRAILS-11687
Current behavior of FlushMode is confusing.  This does not change the story in this post, but maybe will help clarify current behavior.

Added 2014/09/04: 
My next post shows FlushMode behavior in Grails 2.4.3 to be even more confusing.  Example shown in my next post exhibits flush mode behavior opposite to what I get when running the code example from this post.  Seems like Grails 2.3.3 was at least more consistent.

Added 2014/10/25:
Comments on these JIRA tickets:  GRAILS-11797, GRAILS-11536 shed some light on the confusing behavior of FlushMode in 2.4.3.
"It should be noted that HibernateTransaction manager switches the flush mode of the current session to AUTO when the transaction starts."

"Flush mode doesn't have any effect within a transaction. Changes get always flushed at the end of the transaction (a non-read only transaction) if it's not rolledback (the flush mode doesn't matter in transactions)."
The second quote seems to be not exactly incorrect in lieu of examples in my next post.