Over a million developers have joined DZone.

That Moment Where You Should Have Automated but Didn't

DZone 's Guide to

That Moment Where You Should Have Automated but Didn't

One of our MVBs shares his experiences with moving his blog to a central archive, and how he quickly realized automating migrations is never a bad idea.

· Agile Zone ·
Free Resource

On Risk Aversion

Because I’m slightly risk aware I have the following concerns about my main blogs:

  • blogger.com is free so at some point, Google will cancel it and I will lose everything.
  • WordPress uses PHP and a database and at some point, someone will take advantage of a security hole and I’ll get hacked and lose everything.

When I post to LinkedIn I have the concern that:

  • There is no RSS feed.
  • LinkedIn might go down and I’ll lose everything.

And if I ever start posting to medium or other platforms I will have the same issues.

Risk Mitigation

Therefore I decided to ‘archive’ everything into a static site so that if everything goes down I have a backup.

I chose to use Hugo for this.

Subtle Wins

This has the advantage that:

  • all posts in Hugo are written in markdown.
  • I already write all my posts in markdown (and convert them into HTML for pasting into other platforms).
  • I get to use my testerhq.com domain.
  • I get to try out the multiple site hosting on my vidahost cloud plan.
  • It's much harder to hack a static site (and if you do, I just re-upload).
  • I can eventually put all posts on GitHub and have an offsite source backup as well.
  • I can repurpose all the posts later.

I will use testerhq.com as my aggregator and I will probably extend it to cover:

  • What I post to SlideShare.
  • Instagram.
  • Tweets I think are useful.
  • YouTube videos, etc.

But I Didn’t Automate the Migration

I looked around for tools that could easily migrate from WordPress to Hugo, and Blogger to Hugo but didn’t find a lot that fit my workflow.

I thought I should really write my own, but since I saw so many failed attempts out there I thought it might be too hard and take too long (I really should have done that, though).

I thought that since I’ll just migrate once, that I really couldn’t justify the time to automate it (I might migrate each blog once (4 of them), but there are over 450 posts, that might be worth automating).

I Found a Solution

My solution was:

  • Use the WordPress blogger import.
  • Use a Jekyll export Wordpress plugin.
  • Import from Jekyll to Hugo.

I used a Wordpress Bitnami VM and imported my Blogger blogs to that, then exported to Jekyll and then imported to Hugo.

I exported my WordPress blogs directly to Jekyll and imported to Hugo.

And then the pain started.

I Should Probably Have Automated This Part

I then had to check every post for mistakes during the various migrations.

Sometimes it was:

  •   unconverted
  • HTML for iframes of SlideShare and YouTube messed up.
  • Images were not converted to markdown properly.
  • HTML was not converted properly due to a missing tag somewhere.
  • Various special chars not converted properly.

Pretty much every post needed to be hand checked and amended with find and replace in some form or another.

It took me ages. I suspect it would have been faster for me to automate this last part.

I Did Automate Something

I dug out my old HTTP Testing Web Crawler and amended it to check for characters on the page, which made it easy for me to check for:

  • images that hadn’t been converted properly, e.g. and left ![]( type chars on the HTML.

I still have a lot of broken links to stuff I’ve linked to over the years, and I’ll slowly fix that up over time.

I Almost Repeated the Same Mistake

I was looking at an old Choose Your Own Adventure Game that I created a few years ago in Twine with my son. And thought about converting it into more of a CYOA gamebook format e.g.

walk out of the door (goto 8) 

listed on the page rather than a link.

I started hand converting it from a Twine archive into markdown because I looked around and could not find any tools available that did this, and I saw a few failed attempts and I was about to say ‘this is hard.’

But I put together a quick converter using Java and JSoup and it didn’t take long at all.

The benefit is, that because I have automated:

  • I am more likely to do it again.
  • I will probably write more CYOA games.
  • I learned more about JSoup.
  • I have yet more code on GitHub.
  • I re-used the work I put into pandocifier.


agile ,automated ,migration

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}