I will defeat you Grails config loader!

As part of the way we do thing, we wanted to use properties files rather than the plain config.groovy - and thats pretty normal - Grails documentation even gives you a mechanism to load in extra config.

Ideally for us, this could be configured outside of Grails - through some Spring mechanism rather than having to modify the Grails config for every app, but its ok - it has worked. Until today. Today I hit the bug in Groovy where the Properties -> ConfigObject via ConfigSlurper started giving weird results. I had these lines

grails.mail.host = localhost
grails.mail.port = 5000

and i was getting

grails.mail.host = localhost
grails.'mail.port' = 5000
grails.mail.port = [:]

which turns out to be a peculiarity of how Groovy loads the file - and there are quite a few bugs around this on the Groovy bug parade - back since 1.6 (and unclosed).

So how to get Grails to load a properties file in my own fashion? Well thats where the fun started. Turns out that Grails hard codes in its scripts the creation of the DefaultGrailsApplication, which in turn has hard coded the ConfigurationHelper. Neither of these were overrideable or extendable without forking Grails 1.3.7. So one of the things we had been doing was using static methods on our class - and in the end, this turned out to be our saviour - that and Groovy's MOP.

As there were lots of parse methods already, I added a methodMissing to detect for a request for a parse with our class and then passed it onto a method of my choosing that would turn the tree into something more resembling common sense.

That seemed to fix it - so if you have the same problem with broken properties loader when using Grails, that approach worked for me.

Bad request errors from Sonatype's Nexus during release

So we changed the way we published our artifacts to the repository - going straight to the Nexus, instead of via Apache and having Nexus index them. This meant it was easier from a permissions perspective - making sure the right people could upload artifacts into the right places and Nexus and Apache kept mixing up on-disk file permissions.

That unfortunately turned up a very annoying bug in Nexus - you can't attach 0 length files. Typically for artifacts that could generate a WSDL from code, we'd attach them as a -wsdl classifier artifact into the repository. If there were none, then we would touch the file, so the build-helper plugin could be configured to attach the file, otherwise it would fail.

But the move to Nexus knocked that on the head and we could no longer release web service artifacts - so if you run into a situation when you are releasing artifacts and you get a bad request for no good reason - it may be because you are trying to upload a zero length artifact.

The fix was to remove the zero length file and upgrade to 1.6 of the build helper plugin which has a new capability to skip the attachment if its empty. Our plugin definition now looks something like this:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.6</version>
    <executions>
     <execution>
      <id>attach-artifacts</id>
      <phase>package</phase>
      <goals>
       <goal>attach-artifact</goal>
      </goals>
      <configuration>
       <skipAttach>true</skipAttach> <!-- if the file doesn't exist, skip attaching it, added since 1.6 -->
       <artifacts>
        <artifact>
         <file>${project.build.directory}/generated-wsdl.jar</file>
         <type>jar</type>
         <classifier>wsdl</classifier>
        </artifact>
       </artifacts>
      </configuration>
     </execution>
    </executions>
   </plugin>

So I don't forget this

One of the things that that I'm always doing is posting to this blog with the SyntaxHighlighter javascript extension. So to make it easy for myself to remember a bit of code to quickly format the code, I'm posting it here:

'''
code goes <> here
'''.each { c -> print '"><&'.contains(c)?('&#'+(int)c+';'):c }

And it works in the online Groovy console.

Android Tablets

Last year I purchased a Samsung Galaxy Tab 10.1 - I was spending a lot of time browsing the news feeds for stuff for the podcasts, being away from a computer and not being able to make use of valuable time. So I decided to grab a Tablet and see how useful it would be. I was also considering one for use by my wife - given the capabilities of Google Docs Drawing, it was pretty awesome in its ability to do something she was paying for Comic Life on her Mac. I really quite like Android 3.1 and it made moving to my Galaxy Nexus easy and familiar.

It turned out way cheaper than I expected and I have very much enjoyed the experience - unfortunately lately the YouTube has been playing up, but other than that its excellent (seems to be a network issue). The cheap $130 APAD 2 I got was totally useless not because of its resistive screen but because the wifi is utterly lousy, even 1 metre away from the wifi and I can't use it.

The problems I now have with it is that to really use it, I need a keyboard - so ideally I'd replace it with a Transformer Prime with a keyboard - in hindsight I should have waited, but I can always get a keyboard for the Galaxy Tab. The lack of a decent Google Docs client for Android continues to make that application wholly useless unfortunately, so if you are looking for one for that reason, don't bother.

Starting in Maven?

It surprises me again and again when I meet developers who are using Maven and they don't have a local repository. For anyone using Maven, once the decision has been made to actually use it, the next thing you do is get a local Maven repository. Really, its at the top of my list for things to do.

If you are working on projects at home, open source or not, at work with a single developer, whatever, you should install a Repository Manager. If you have more than one developer using Maven, you are simply insane not to use a one. So why would you install a repository manager even if you are only a single developer? Surely its hard?

Well, no, it is incredibly simple, Sonatype's Nexus takes 10 minutes to set up for example. Here are a few if you don't already have one:

• Sonatype's Nexus - this comes in a Pro and Free version. I use Free in three different places and it has served me very well. 
• Apache Archiva - I used this in my last job, and I didn't like it as much as I like Nexus. It seemed a little more complex to set up. 
• JFrog's Artifactory - I haven't used this, but they do a lot with the Gradle community and are going to do some work around Grails plugins. 

So back to why? A few reasons

• Unfortunately, not everything is in Maven Central. This means you have to edit your settings.xml to add each repo in as you need it. Don't, put them in your Repository Manager and just point to it - let it sort them out. Add a new repo? Just add it in the Repository Manager. 
• If you are doing Maven releases properly, you are using version ranges. Version ranges cause Maven to scan all repos in your settings.xml for updates on a regular basis - and you don't want this traffic going far if you can help it, it slows you right down.
• If you don't have a Repository Manager, you are either (a) not releasing you artifact (a cardinal sin) or (b) only releasing to central (which is feasible for some projects, e.g. easyb). 
• It allows you to blow away your local repository when it gets untidy from snapshot builds which people use periodically. Note if you do everything using snapshot builds, you are using Maven in a  broken fashion.
• You can put a Repository Manager on your own machine as a proxy as well, if you want to keep as much network traffic local for example, but you have the main Repository Manager hosted elsewhere - on a slow VPN or in another country.

Releasing

I've known people working on projects who have never released - they just work with snapshots! Thats nuts! A release in maven is an individual artifact (your entire war would be one, as well as an individual api) - it is something that has been cleaned, started from scratch, compiled, verified, all tests run, tagged in the remote repository, checked out again to a different location and all  that done again and then uploaded to your Repository Manager. You should be releasing very regularly. And be fine grained about your artifacts! As soon as you have done a significant lot of work, release it. 

Version Ranges

Version ranges are important for two main areas of Maven usage. 

• You have chosen an open source project based on its functionality, license, code quality, whatever. But you choose a specific version. For example, in a recent project we chose SOLR - 3.3.0. Our use of the 3.3.0 means we use <version>[3.3.0]</version> as our version. This means I want 3.3.0 and nothing else. Not 3.3.1, not 3.3.5, not 3.4.0. Why is that important? SOLR when it released 3.4.0 caused the meta data in (Apache) Maven Central to get corrupted - this means the only version available was 3.4.0. If I had instead specified <version>3.3.0</version> - if Maven didn't have it locally, it would go pick up 3.4.0 and happily use it. If the corruption had caused only 2.3.0 to be available, Maven would have used that - it would have been a mess! Always specify exactly what ranges you are  happy to include artifacts for - if stuff breaks, lodge JIRA tickets on Sonatype's JIRA, they are pretty quick about it and will be ensured actually using what you have chosen. 
• Generally projects decide on how they issue a new version number - some just use two (major.minor), where the major changes on a breaking api change, some use three levels. You should specify a range when you are happy so new releases turn up - and typically you will do this of your own artifacts. You start at version 1.1 (because thats how ranges work, don't start at 1.0 or 0.x, thats just nonsense) and keep releasing until you break. Release early and often. If someone then who is using your artifact says [1,2) or say [1.7,2) (if 1.7 is the start of your api having what they want) then they will just get new releases as they come out, ensuring they keep current. These kinds of version ranges make Maven use really nice and easy to use when it comes to dependencies (along with composites, another post). 

Composites are just pom's that contain a collection of dependencies - we have composite-cxf, composite-spring, composite-jackson, composite-logging-api, composite-logging-api, etc. Composites keep your pom clean, they take out all of the stuff that isn't really specifically to do with your App, but the kind of App you are building. They become accepted building blocks - and they rev. like normal artifacts. Changing from Spring 2.5 to Spring 3.1? rev the major version of your pom - existing artifacts won't be broken, you can fix them as necessary. Composites can be great for public facing releases as well - if Wicket 1.6 was just a pom containing fine grained artifacts that were hard version ranged, then we they released bug fixes (early and often), you could override specific artifacts and bring in the new releases to get the ones you want. 

And another cardinal sin - it happens, but rarely, do not put dependencies in parent poms. Except for the plugins which often need them (and try and use composites for them), but your parent poms should not have modules or dependencies defined. Put them in the composites. 

(It sometimes happens when you are using projects that are so terribly broken you can't help it, but its rare)

Do yourself a favour, if you are using Maven, install a Repository Manager. 

Initializing your Grails Services

In one of my current Grails projects I need to do what I would normally use @PostConstruct for, but am unable to as usually GORM has not completed its additional methods by the time it comes to use them. So the general recommendation is to do it in the bootstrap - a technique I generally do not like as it means you cannot make it a best practice easily - just include a dependency and its always done.

Ideally I think I'll need to turn this into a plugin, but I hate source code plugins enough to just include the Bootstrap at the moment.

So this goes in your Grails conf, and its a slight modification of the one you find on Google - it checks the scope of the bean, singleton ones only are initialized.

import org.codehaus.groovy.grails.commons.GrailsClass
import org.codehaus.groovy.grails.commons.GrailsClassUtils

class BootStrap {
  def grailsApplication

  def init = { servletContext ->
    grailsApplication.serviceClasses.each { GrailsClass gClass ->

      String scope = GrailsClassUtils.getStaticPropertyValue(gClass.clazz, "scope")

      if (!scope || scope == "singleton") {
        def serviceBean = grailsApplication.mainContext.getBean(gClass.propertyName)

        if (serviceBean.metaClass.respondsTo(serviceBean, 'grailsInit')) {
          serviceBean.grailsInit()
        }
      }
    }
  }

  def destroy = {
  }
}

Grails and Maven - blue grails

One of the things that has kept me away from Grails for a long time - even after spending a couple of years knee-deep in it has been the extremely poor support for a clean build and release.

The combination of Gant scripts and source-code plugin dependencies, the use of Ivy as a dependency manager, endless outstanding bugs and a tendency to just not understand why this was so important (I'm sure everyone had their list) led me to abandon using Grails or pushing Grails in any real way in my work life while working at Sky Television (NZ). I still used it in a side project, but not in any anger and long since left the developers discussion list.

As I prepared to take on a new job doing Grails development again, going back from an environment that focused on neat, tidy granular artifacts that were tested and released, wired cleanly with Spring and assembled with clarity and speed was something I didn't want to loose. I learnt a lot from the two guys I worked with in terms of the use of Spring, Maven and API design. I only hoped that Grails would be able to live up to it now it had moved on.

On jumping back into Grails I saw that essentially little had changed (even if it was about to with 2.0 and binary plugins). Application design still seemed to encourage big single projects where all controllers, domain objects, services and so forth were all in the one project, making it harder and harder to test, longer and longer to build and deploy. Knowledge of Spring by users was limited, consequences of Hibernate ignored, discipline around tagging and releasing artifacts almost non-existent. EDIT: I mean this in the context of the Grails community as a whole:

• Spring encourages the use of interfaces. Grails encourages you to inject the service's actual class by its name (e.g. userService for com.bob.UserService) but by using a def you can mock it. Its Groovy, but its nasty.
• Grail's finders ignore the consequences of the SQL generates and most developers appear to not look at the SQL Hibernate generates. Hibernate does a bad job with anything thats not basic, and createCriteria is a demon. Whether this is better or worse than any other usage of it I'm not sure.
• Grails doesn't come with a "release" script - crazy in my opinion. The concept of clean, build, tag, checkout from tag in a different place, build is critical to knowing you have a reproducible clean build. Not having it is crazy and is an artifact of people who don't use Maven, or one of the many who use Maven who don't use properly.

So for the past few months I have been working with the University of Auckland and TEIQ to build a set of best practices around modular Grails development, using Maven in the sanest way possible (and it is sane, and nice, and clean). Sorting out the way we deal with the inevitable buggy plugins (including the Grails Maven Plugin), building composites that bundle together packages of artifacts that relate, standardising web services structures and looking always for opportunities to make things better and cleaner, faster and clearer, better tested and more robust. Its a great opportunity and I thank TEIQ and particularly the University for giving me the chance to make a difference. And great people - did I mention great people?

As part of this work, we are taking some of what we have learnt and pushing it out into open source - or probably more open documentation. To make the best use of what we have done, we really need to have the Grails plugins that we use available as Maven artifacts - and I can't see the Grails guys happy about us doing that. I'm going to document it here on my blog - to make sure I do it (document it that is) - I'd recommend taking what I write if it works for your team and putting it in your Wiki and augmenting it over time.

The first thing I'm putting out there which may be of some worth to people who come to Grails and consider using Maven is my set of scripts I use. Its growing - but it provides me with a great set of shortcuts to the commands I regularly use. Enjoy :-)