Liquibase: Git for the Database

I’ve used the tag line “Liquibase: Source Control For your Database” for a few years now, despite the fact that developers are always on the lookout for categorization errors and love to argue that it’s not really a source control system because bleah, bleah, bleah. 

Nevertheless, I’ve always seen it as a good analogy despite the differences. In fact, while early versions of Liquibase used the term “migrate” as the command to apply un-run changes, I changed the command to “update” in the early 1.0 days to better match the terminology of the state-of-the-art (at the time) version control, Subversion.

So why not just use SVN or Git or any other “real” version control system for your database? Because how you manage your code and how you manage your database have a fundamental difference—it doesn’t matter how the text of your code goes from point A to point B but if the data in your database takes the wrong path from point A to point B it can have fire-able consequences. Therefore, what Liquibase specializes in is letting you specify those steps and ensuring they are executed as needed against all your databases. Those changes are stored in a text based human-readable format so that you can manage them in whatever “real” version-control system you have.

So what is the usual Liquibase workflow in terms of a version control system like Git?

Start a new project: “create an empty changelog file” aka “git init”

If you were creating a new Git-managed directory, you would run “git init” to initially set up the repository. There is no special Liquibase command to create a new database changelog, you just create an empty XML (or YAML or JSON if you prefer) file.

Start Coding: “add changeSets” aka “git add”

As you create new source files, “git add” adds the files to the index so they will be included in your next commit. In Liquibase, you add new changeSets to the databaseChangeLog file describing the steps you need done in the order they should be executed.

<databaseChangeLog
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">

    <changeSet id="SAMPLE_1" author="alice">
        <createTable tableName="employee">
            <column name="id" type="int" autoIncrement="true">
                <constraints primaryKey="true"/>
            </column>
            <column name="first_name" type="varchar(255)"/>
            <column name="last_name" type="varchar(255)">
                <constraints nullable="false"/>
            </column>
        </createTable>
    </changeSet>

    <changeSet id="create address table" author="bob">
        <createTable tableName="address">
            <column name="id" type="int" autoIncrement="true">
                <constraints primaryKey="true"/>
            </column>
            <column name="line1" type="varchar(255)">
                <constraints nullable="false"/>
            </column>
            <column name="line2" type="varchar(255)"/>
            <column name="city" type="varchar(100)">
                <constraints nullable="false"/>
            </column>
            <column name="employee_id" type="int">
                <constraints nullable="false" foreignKeyName="fk_address_employee" references="employee(id)"/>
            </column>
        </createTable>
    </changeSet>

</databaseChangeLog>

Save the work you’ve done: “liquibase update” aka “git commit”

With Git, when you’re at a good stopping point, “git commit” saves your changes to your local repository. “Liquibase update” will apply your new changeSets to your local database. Once could argue that “liquibase update” is a better match for compiling your code because it is something you should do after adding each changeSet to ensure your logic is correct. Nevertheless, running “liquibase update” will look at each changeSet in turn and check the database’s DATABASECHANGELOG table to determine if it has ran or not and will only run new changeSets.

Share your work with the rest of the team: “git add; git commit; git push” aka “git push”

Once you’ve tested that your code and/or database changes work locally, it’s time to send your changes to the rest of the team. With standard Git, you’ve already added and committed your code, so you just push your commits. So far with Liquibase, however, you have just been editing a file so you need to do a standard “git add” and “git commit” on the file before “git push”-ing it along with the rest of your codebase.

@#%$&ing Conflicts: “git merge” aka “merge hell”

Just like with code, you will run into code conflicts. This is where it is valuable to have the changelog file be a human readable text format. Git and other version control systems can often seamlessly merge your changes in, but if and when it cannot, your standard merge tools will present you the opportunity to combine the changes.

Normally changelog conflicts can be resolved simply by ensuring that all your changeSets and all their changeSets are included because you have divided up the work and are all working in different areas of the database. However, there are times when you both try to add the same table or both alter the datatype of an existing column to different types. In these cases, you choose which changeSets make the most sense to keep and/or create new changeSets that combine the problem ones. Depending on how widespread the conflicts are across your databases, you may want to use liquibase preconditions or contexts to control where merged changeSets are executed.

Branching: “git branch; git merge” vs. “git branch; git merge”

Development is never straightforward, which is why branches are so invaluable. This applies to your database as much as to your code. Liquibase is designed to handle branches by tracking each changeSet individually by a combination of the id, the author, and the filename. Earlier database versioning systems would simply use an incrementing version number or a timestamp which would run into problems when multiple branches both try to add a version “42”.

With Liquibase, you lose the nice version number and instead just know that changeSets “nvoxland:3113:com/example/changelog.xml”, “nvoxland:421:com/example/changelog.xml” and “fred:421:com/example/changelog.xml” were executed. Think of it like how Git had to lose the simple SVN version numbering in favor of versions like “95ccad016ede5ded704203e632b59cc1417957c2” in order to handle distributed branches.

When you need to make a branch, simply branch the changelog file in your version control like everything else. As you append changes in your branch, those changes will be applied to just your local database. When you merge the changelog file back into the base branch, those new changeSets may end up higher in the changelog file than others that are already applied to other databases, but that is OK because each changeSet is checked individually and the newly merged changeSets will still be applied to all databases in their correct order.