Grails memory leak on batch inserts and a possible OOM

Recently, I worked on downloading a subset of our huge database into a grails' HSQL. Initially, I thought this was so easy that it almost qualifies as hello world2.

But I hit a roadblock. The job that downloads rows and saves to the HSQL would go out of memory after a 100K rows or so. Specifially, it would throw "java.lang.OutOfMemoryError: Java heap space". I was certain that this was memory failing to get recollect. But without going through the memory analysis, I went after obvious suspects: hibernate session and HSQL transactional memory. I took clues from here and here. But no avail. Finally, this post from Burt Beckwith explained it all.

All I needed to do was set the 'errors' to null after saving domain objects.

To summarize the leak: Grails keeps references to the errors objects that are on the domain objects in the httprequest. So, as long as a request is around, the domain objects will not be garbage collected. Makes sense? :)

Lessons learned : Grails upgrade from 1.2.1 to 1.3.6

Recently (due to the downtime provided by holidays), I upgraded our main project to grails 1.3.6 from 1.2.1. It wasn't a smooth ride. Here are some lessons learned (I'll continue to add):

1. Don't wait too long before grails upgrade. Do an upgrade at your first opportunity. That saves you from being on your own island.

2. If you use groovyws jar, make sure you exclude xmlbeans and jaxen. These are available in jdk1.6. This wasn't a problem in older grails; the classloading has probably changed.

runtime('org.codehaus.groovy.modules:groovyws:0.5.2') {
excludes 'xmlbeans', 'jaxen'
}

3. I am currently trying to find out why the integration tests are not rolling back the data inserts. I'll put the solution here once I know it.