java.time for the extremely impatient

This is a super quick history and primer for those familiar with java, but unfamiliar with Joda-Time.
Java’s original date and time handling had some issues that could make it cumbersome and error-prone to work with. Then came the Joda-Time library, which is a fantastic replacement for pretty much everything date and time related. It’s intuitive, clean, fully-featured, and performant.

Starting with Java 8, the library was folded into the core JDK, with very few modifications to the Joda API.

In short, if you’re not on Java 8 yet, you should probably be using the joda-time library. If you are on Java 8, you should use the java.time classes.

There’s much more to the package than what I’ll show, and you’ll eventually want to dive deeper, but this article shows you some of the most frequently used techniques I’ve experienced on the last few projects. I’m using groovy for the code snippets, but they’re just java without the semicolons.

91.83% of the time, you’ll work with LocalDate or ZonedDateTime.

import java.time.LocalDate
import java.time.ZonedDateTime
import java.time.ZoneId
import java.time.format.DateTimeFormatter
import java.time.temporal.TemporalAdjusters

// If you have a date without time information, use a LocalDate.  
e.g. someone's birthday
LocalDate localDate = new LocalDate(2016, 4, 12)
println localDate.toString()  // 2016-04-12

// If you need to include time in the date, use ZonedDateTime
ZonedDateTime zdt = ZonedDateTime.now() 

// when formatting a ZonedDateTime for API communication, 
// you'll typically use the DateTimeFormatter.ISO_INSTANT format
println zdt.format(DateTimeFormatter.ISO_INSTANT)  
// 2016-04-12T19:20:45.539Z

// some more examples of formatters
println zdt.format(DateTimeFormatter.ISO_LOCAL_DATE_TIME)  
// 2016-04-12T14:20:45.539

println zdt.format(DateTimeFormatter.RFC_1123_DATE_TIME)   
// Tue, 12 Apr 2016 14:20:45 -0500

println zdt.format(DateTimeFormatter.ISO_ZONED_DATE_TIME)  
// 2016-04-12T14:20:45.539-05:00[America/Chicago]

// you can also create a custom formatter
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("MMM dd yyyy GG")
println zdt.format(formatter)  // Apr 12 2016 AD

Date manipulation is fluent and intuitive.

import java.time.LocalDate
import java.time.temporal.TemporalAdjusters

LocalDate localDate = LocalDate.now()
println localDate 
// 2016-04-12

println localDate.plusMonths(1).withDayOfMonth(1) 
// 2016-05-01

println localDate.minusMonths(1).with(TemporalAdjusters.lastDayOfMonth()) 
// 2016-03-31

Prefer to work in UTC if you can - logs, database timestamps…consistency helps avoid mistakes.

in a startup class:
TimeZone.setDefault(TimeZone.getTimeZone("UTC"))
or as a JVM flag:
-Duser.timezone=UTC

When working with a point in time, always be aware of timezone!!!!

Even when working with objects like LocalDate, you must be timezone aware if you’re using a point-in-time operation. And by point-in-time, 99.273% of the time I’m referring to the now() method.
For example, given the following scenario:
- The default JVM timezone is UTC.
- At 2016-04-12 3:10 PM in America/Chicago (which is GMT-6), a call to
println LocalDate.now() will result in 2016-04-12.

But at 2016-04-12 6:10 PM in America/Chicago, the same call will result in 2016-04-13

If you’re in Chicago, that looks like tomorrow, and might be a bug for what you’re trying to do! The code is doing precisely what it’s told though; the date in London is April 13th at the time the LocalDate is created for the UTC timezone.

So to repeat, when working with specific points in time, be aware of the timezone you’re working with. For example, if you want to set a date 30 days out from now according to your business location, you could say LocalDate.now(ZoneId.of('America/Chicago')).plusDays(30).

Further Reading

The java.time javadocs are actually really good, so read them for more detailed information. Wrapping up the whirlwind tour, here’s a quick table to help get you started with a few objects and their example uses:

Object	Example Usage and Notes
LocalDate	Birthday, Contract date where no time or timezone is needed
ZonedDateTime	Most points in time, like some startDateTime, endDateTime
YearMonth	When you only want to work with a year/month combination. Less commonly used, but handy for date comparisons if you don’t want granularity to the day. e.g. Credit Card Expiration Month
LocalTime	e.g. Chris’s hardware store opens at 8 am. This date is irrespective of timezone (i.e. you wouldn’t change the opening time when daylight savings rolls around.)

Dynamic Grails Tomcat datasource configuration with Etcd

Ever wonder if you could modify a Grails datasource while the app is running?
Probably not, and that's totally fine...most people don't need to.

We had a couple reasons though:
1. During a disaster recovery situation where a non-clustered database goes down, you want to point all the apps at a failover database.  By default this means you have to update the config and restart all the apps.  On a typical AWS instance, this means at least a minute of downtime for a bigger Grails app.  Not the end of the world, but not great.

2. One of our databases is a catalog of product information that can be drastically changed.  We wanted to be able to clone the catalog, apply massive data changes to it (this can take a minute or so), and then point all the apps in the cluster to this new database without downtime.  And we also want to be able to revert to the old database if something goes wrong.

First question - how can you change a Tomcat datasource while the app is running?

Grails version 2.3 and onwards uses the Tomcat Connection Pool as its datasource provider by default.  If you're not using Grails 2.3+ yet, you're probably using the Apache Commons DBCP, and can switch by using this plugin.

Basically, you pass the ensureCurrentDatasources method your grailsApplication and a list of datasource names you want to inspect for changes and potentially refresh.  The datasource name(s) are typically defined in your DataSource.groovy.  e.g. If you only have one datasource, it'll be named "dataSource".  If you're using multiple datasources, they might be named "dataSource_auditing" or whatever you've specified.

The method is implemented to compare the current Tomcat connection pool values for the username, password, and url against the current Grails configuration values.  If any settings have changed, it'll update those connection pool settings and call the purge() method in the connection pool.  purge() will basically perform a graceful reset of all the connections so that they establish their next connection with the updated configuration.  I chose username, password, and url because those are the things that we might change.  There are more properties in the pool that you could possibly change, but you probably don't want to change much else, since there is some critical state being managed by some of the properties.

OK, so you know a way to dynamically update a datasource while the app is running.

Next question: How should I wire in this dynamic update capability?

The short answer is, whatever works best for you.  Here's the path we went down...

The initial approach:
Our application has the following attributes:

• It uses an inline plugin where we keep our domain classes and services.
• We use SaltStack to manage our external config files
• It uses the External Config Reload plugin to allow us to dynamically update the app config when we change the config files.

With those attributes, we initially implemented a hook into the TomcatDatasourceUtil by defining the onConfigChange event in our plugin's Config.groovy, like this:

def onConfigChange = { event ->

    TomcatDatasourceUtil.ensureCurrentDatasources(

            application, 

            ['dataSource', 'dataSource_auditing'])

    }

This worked fine, but seemed like a clunky solution.  For the catalog database update scenario, the application essentially needs to remotely communicate with salt, so that salt could remotely update all of the application's configuration files.  We keep all our salt configurations in source control, which didn't really fit the model of what we wanted to do.

   
The better approach...or at least this has been working well for us so far:
Rather than use a tool to constantly push out config file changes on the fly to our cluster of apps, we thought it would be better if we inverted the technique...i.e. have all the applications get their configuration from a central location.  This is where etcd comes in.  The summary of etcd is that it's "a distributed, consistent key value store for shared configuration and service discovery with a focus on being simple, secure, fast, and reliable."

You can run just about any groovy code in your Config.groovy and Datasource.groovy.  So rather than have the application get its datasource config info from a file, we have it load the datasource URL from etcd.  e.g. Here's a snippet from our external config file:

This will take care of your app getting its initial url value from etcd.  You can put whatever else in etcd that you want...for our case we only need to dynamically change the url.

So now how do you update the datasource for a cluster of applications?
In your application's Bootstrap.groovy init, make a call to a class like this:

This is a very basic implementation of an etcd client that can watch for changes, update the grails configuration upon change, and also allow the app to update an etcd value.  There are more robust etcd clients available, but we didn't need (at least not yet) the added dependencies and complexity.

It's pretty fun to watch once you get it all working.  Essentially this is the flow:
1. A cluster of grails applications start up, configure their datasource URL using the etcd config, and watch for changes.
2. Some time later, one of the applications clones the database, makes changes to it, and then sets the new URL value in etcd.
3. All the applications are then notified of the updated etcd value and dynamically update their datasource to point at the new URL.
4. "dataSource was modified and refreshed"!

Does this actually work?
We've been running with this technique for a few weeks in production now, so I wouldn't call it extremely battle-tested yet, but we haven't seen any problems yet.

A couple thoughts to go along with this:
- build the dynamic datasource refresh capability as a plugin.  maybe, but probably not a ubiquitous problem.
- put all of our configuration in etcd.  this is interesting, and we might eventually, but we're going to watch this solution for a while first.  I don't mean to bash salt at all - managing config files with salt has worked well for us so far; it just didn't fit this scenario well.
- support other types of connection pools.  All the cool kids these days are using HikariCP.  I haven't used it yet with Grails, but do for Clojure projects.  I don't know if it supports the purge concept yet, and if Grails will play well with it, but it might be an even better solution.

Update 8/22/2015: This initial solution has worked fantastically in the last five months, without needing any changes yet.  I had expected there would be some hiccups in its stability (e.g. maybe needing a more robust etcd client), but we haven't had an issue with it yet.

Groovy collect vs spread-dot operator

Yesterday I was doing some Groovy code cleanup with the wonderful CodeNarc static analysis tool. One of the violations it found was UnnecessaryCollectCall.
The summary of the rule is "Some method calls to Object.collect(Closure) can be replaced with the spread operator."
e.g. Replace things.collect { it.name } with things*.name, or even things.name if what you're after is a property.

But when I performed that refactoring and ran all the tests, some failed! Here's why:

I made the mistake of assuming that the spread operator behavior is always identical to the collect method. For a non-null collection, it is. e.g. The following code will produce the same result regardless of the technique you use:

Loading ....

But if the collection you're operating on is null, the three techniques will result in different outcomes:

Loading ....

What this means is that if you're working with Collections that can potentially be null, you need to think about the consequences of the dot operations before using them. i.e. Don't ever use the implicit spread operator (things.a) if the collection can be null. And only use *. if it's ok for the result to be null.

tl;dr; Explicit and implicit spread operations are great, but be aware that they are less forgiving than the collect method

Useful Parallels Windows VM Keyboard Shortcuts

Summary: If you use a Windows VM in Parallels, add these extra keyboard shortcuts in the preferences to make your windows home/end and selection experience more consistent with OS X:

For example, by default in SQL Server Management Studio in a VM, the keyboard shortcut for Home is Fn+Shift+Left.  That's not a standard shortcut though.  The normal keyboard shortcut for Home in OS X on a keyboard without a dedicated Home key is Cmd+Left.   But the default key mapping for Cmd+Left in the windows guest is to move the window.  Argh!

After doing this a dozen times, I finally opened up the Parallels preferences and saw how easy it was to add a custom key mapping, as indicated by the four shortcuts outlined above.  Enjoy!

MyBatis Migrations with Grails

I currently prefer using MyBatis Migrations for substantial Grails apps.  In the world of Grails, this is an unconventional choice, so it wasn't made lightly.  But after having used it for a while on a moderately large app, it still feels like the right choice for us.

Background

As you probably know, the official Grails database migration plugin is based off of liquibase.  I've tried using it a couple times in the past on other projects, including non-Grails apps, but it wasn't ever a good fit from a technical or stylistic point of view.  When it was made the official migration style of Grails, I gave it another shot, but too many things still got in the way compared with MyBatis Migrations.

What pushed me away from Liquibase and the database-migrations plugin

• Like all software abstractions, there are leaks and bugs.  e.g. Do a search for your favorite database plus the word "liquibase issue".  The point isn't to criticize the software or its authors - they're master craftsmen.  The point is that you can eliminate those issues if you eliminate the abstraction itself.
• In my project's culture, the DBAs want to review and apply the database changes in the form of SQL - not in the form of XML. This is a fair requirement in my opinion. You can use the dbm-update-sql command to generate SQL scripts, but this is problematic for two reasons. First, you can only run the command for a target environment that hasn't been migrated yet. Usually this is the opposite of what I want to do...I want to create a script for migrations that I've already created and tested. Second, DBAs may want to tweak the resulting SQL, but that means the production environment is different than the test and development environments. 
• I like the potential of being able to use groovy for migrations, but there are gotchas in this approach that I also experienced.  Again, I can avoid this issue entirely if I treat the database as a separately managed entity, rather than try to treat it like an extension of a programming language.
• The liquibase and groovy DSL is not portable from a development sense.  e.g. You can create manual SQL in the migration like this
  sql("update foo set bar = true")
  but  you can't easily copy/paste that to and from your favorite SQL development IDE during your development experimentations. 

Along with avoiding the deficiencies mentioned above, here are some more reasons why it works well for us.:

• It's simpler. There are 9 available commands in MyBatis Migrations, compared with 31 in the database-migrations plugin.   You're writing straight-up SQL using a simple framework.
• It's faster.  Running "bin/migrate status" takes about 0.6 seconds on my machine (because it's just a shell script execution), whereas running "grails dbm-status" takes about 6 seconds (because it's firing up the grails environment).  

Basically, it gives us the full control of our database design and migrations that we need without getting in the way.

How We Use MyBatis Migrations

A note that's not specific to MyBatis Migrations or Liquibase...I only apply "real" migrations to a project after it's settled down. i.e. I work with the H2 in-memory database with create-drop for as long as I can when I start a new project. I don't care what the database looks like during the the early stages of development, and auto-creating it gives monster productivity gains. When data needs to start sticking around (e.g. in the user acceptance test environment), then I'll turn off automated migrations and use a more manual approach.

Here's what our project structure looks like:

├── db
│   ├── bin
│   │   ├── migrate
│   │   └── migrate.cmd
│   ├── db-creation
│   │   ├── init-db.sql (custom script to initialize each database)
│   ├── drivers
│   │   └── sqljdbc4-4.0.2206.100.jar
│   ├── environments
│   │   ├── ci.properties
│   │   ├── development.properties
│   │   ├── functional.properties
│   │   ├── qa.properties
│   │   ├── test.properties
│   │   └── uat.properties
│   ├── lib
│   │   ├── mybatis-3.1.1.jar
│   │   └── mybatis-migrations-3.1.1.jar
│   ├── scripts
│   │   ├── 20120609160926_create_changelog.sql
│   │   ├── 20120611201651_create_schemas.sql
│   │   ├── 20120611211651_initial_add.sql
│   │   ├── ...
├── grails-app
│   ├── conf
...and the rest of the grails conventional directories

The db folder is in source control with the rest of the app.  In Bootstrap.groovy, we go down to the create_changelog script and back up to the top for our automated test environments (test, ci, and functional).  e.g.:

We deploy the migrations by packaging the db folder with each release. We then have the choice of either automating the migrations to run when the app is deployed, or running the migrations ahead of time (when it's a backwards-compatible db change).

tl;dr - Using MyBatis Migrations works great for a grails app when you're comfortable being in full control of all the database changes.

Groovy kid's addition test generator

One of my munchkins started doing weekly math speed tests at school, where they have 2 minutes to try to answer 40 addition questions. To make practice more fun, I bought a great little ipad app called MathBoard. The app really is nice, but it's still not the same as pencil and paper, so I looked online for a way to generate the tests that I could print. I waded through about a dozen of them, and they all failed for one reason or another - wouldn't do 40 questions, poorly formatted, include extra garbage, don't print on one page, etc... So I spent a few minutes and created this little groovy script:

Loading ....

How to Override Groovy Instance and Static Methods

Here are some examples on how to inject behavior into Groovy object methods. In the sample class below, method1 is an instance method with no arguments. method2 is also an instance method, but with a single String argument. method3 is a static method, which requires a slightly different way of injection.

Output from running groovy Test.groovy:

class Test {
    void method1() {
        println 'In method1'
    }

    void method1(String foo) {
        println "In method1 with String param of $foo"
    }

    static method2(String foo) {
        println "In static method2 with String param of $foo"
    }

    public static void main(String[] args) {

        def test = new Test()

        test.method1()
        test.method1("asdf")
        Test.method2("asdf")

        MetaMethod method1NoParamMethod = test.metaClass.pickMethod("method1", null as Class[])
        test.metaClass.method1 = { 
            println "augmenting no param method1"
            method1NoParamMethod.invoke(delegate)
        }

        MetaMethod method1StringParamMethod = test.metaClass.pickMethod("method1", String)
        test.metaClass.method1 = { String fooParam ->
            println "augmenting String param method1"
            method1StringParamMethod.invoke(delegate, fooParam)
        }

        MetaMethod method2Method = Test.metaClass.getStaticMetaMethod("method2", String)
        Test.metaClass.static.method2 = { String fooParam ->
            println "augmenting static method2"
            method2Method.invoke(delegate, fooParam)
        }

        test.method1()
        test.method1("asdf")
        Test.method2("asdf")

    }
}

In method1
In method1 with String param of asdf
In static method2 with String param of asdf
augmenting no param method1
In method1
augmenting String param method1
In method1 with String param of asdf
augmenting static method2
In static method2 with String param of asdf