Monday, April 30, 2012

Critical failures of Apache Maven Central

by Richard Vowles

Today's failure of the meta-data of logback really hit us hard and resulted in significant downtime. What was the issue? Its an issue that appears to happen relatively frequently - someone releases a new version of their library and the release process blows away the Maven metadata - making it appear to anything checking the meta data that there is only one version available - the current one.

As at time of writing, the Maven metadata for logback-classic looks like this:










We are using the version range [0.9.17] for Grails 1.3.7 projects and [1.0.1] for Grails 2.0.3 projects. This is good Maven hygiene - we want the build to fail if these specific versions are not available for a good reason (such as no repository available) - not for a bad reason like someone broke the repository. We don't want Maven to choose a different version, we want that specific version.

When you use version ranges, you need the meta data - that tells Maven what versions are available. If you don't specify a range, it shouldn't use the meta data. However... With Maven 3.0.4 today, that turned out not to happen for some reason.

Whats worse, is that when we wanted to work around the problem by relaxing the version ranges, it didn't work. It just got worse and weirder. Eventually, on the brainwave of Michael McCallum - installing the artifact in our 3rd party repository and moving it ahead of Central in the Nexus allowed us to get back to work.

Now if this was the first time this happened, it wouldn't ring such alarm bells, but it isn't. Its at least the third (SOLR being the first time we hit it). I'm now in the unenviable position to have to discuss with my team whether direct Apache Maven Central access will be banned. If the artifact isn't taken from Central and put into our 3rd party repo, it can't be used. We just cannot afford the downtime.

What I don't understand is why there is no automatic process for repair for Central. Its a critical resource, which we enormously appreciate but its value in accessing directly has become much lessened after today's extreme waste of productive, valuable time.

Update: Another one -

Friday, April 27, 2012

Code Lounge, Post Mortem + Hangout Lounge

by Richard Vowles

We held our first Code Lounge on Wednesday - Anzac Day in Auckland. We went to the morning service at Glendowie College and then proceeded a rapid clean up to ensure we were ready for the three people arriving. I've posted elsewhere about the results (ZeroMQ vs RabbitMQ vs Pusher), the day went pretty well in my opinion - we stopped every hour and made sure we were aware of what the other team was doing - we set ourselves goals which we overstepped periodically and when we got to the end of the day (3pm) and had only an hour left, we decided whether we continued learning (David and Irina decided to continue getting the Clustering working for Rabbit and trying to work out its fail over strategy) and Mark and I decided to push on with Pusher (which turned out to be an exercise in re-writing the library instead).

All in all a good day. But I was lying on the bed after an exhausting week while listening to my wife practice Cello (which is a surprisingly meditate experience) and thinking about my backlog of books to read. They consist of programming language books (Dart 4 Hipsters for example), Tool books (a few around VIM are bubbling around, Pragprog have just released a new early access), Electronics books (MSP430 and various others), General informational/idealogical books (Information Diet, Biohacking) and fiction. And I'm not really reading them, particularly the programming language ones and I really really want to.

So I'm going to get inspired by Chris Strom (Dart 4 Hipsters, The Spdy Book, Backbone.js recipes) and try starting chains ( and see if I can just knuckle down and self educate.

I spoke with Mark about it tonight and wondered aloud whether or not having a Hangout open when I was working away at a particularly topic might be a worthwhile idea. I think I will do it, I will try just having an open hangout around  a particular topic and ensure I circle Programming related people in my G+ stream. If they want to come in they can otherwise I at least will continue to learn. I'll set a goal for 1 hour a day and see how I get on.

(PS Manning have a great special on at the moment which includes a lot of interesting books, I'm trying desperately to not go and buy them and have them sitting there as well!)