That Moment Where You Should Have Automated but Didn't
One of our MVBs shares his experiences with moving his blog to a central archive, and how he quickly realized automating migrations is never a bad idea.
Join the DZone community and get the full member experience.
Join For FreeOn Risk Aversion
Because I’m slightly risk aware I have the following concerns about my main blogs:
blogger.com
is free so at some point, Google will cancel it and I will lose everything.- WordPress uses PHP and a database and at some point, someone will take advantage of a security hole and I’ll get hacked and lose everything.
When I post to LinkedIn I have the concern that:
- There is no RSS feed.
- LinkedIn might go down and I’ll lose everything.
And if I ever start posting to medium or other platforms I will have the same issues.
Risk Mitigation
Therefore I decided to ‘archive’ everything into a static site so that if everything goes down I have a backup.
I chose to use Hugo for this.
Subtle Wins
This has the advantage that:
- all posts in Hugo are written in markdown.
- I already write all my posts in markdown (and convert them into HTML for pasting into other platforms).
- I get to use my testerhq.com domain.
- I get to try out the multiple site hosting on my vidahost cloud plan.
- It's much harder to hack a static site (and if you do, I just re-upload).
- I can eventually put all posts on GitHub and have an offsite source backup as well.
- I can repurpose all the posts later.
I will use testerhq.com as my aggregator and I will probably extend it to cover:
- What I post to SlideShare.
- Instagram.
- Tweets I think are useful.
- YouTube videos, etc.
But I Didn’t Automate the Migration
I looked around for tools that could easily migrate from WordPress to Hugo, and Blogger to Hugo but didn’t find a lot that fit my workflow.
I thought I should really write my own, but since I saw so many failed attempts out there I thought it might be too hard and take too long (I really should have done that, though).
I thought that since I’ll just migrate once, that I really couldn’t justify the time to automate it (I might migrate each blog once (4 of them), but there are over 450 posts, that might be worth automating).
I Found a Solution
My solution was:
- Use the WordPress blogger import.
- Use a Jekyll export Wordpress plugin.
- Import from Jekyll to Hugo.
I used a Wordpress Bitnami VM and imported my Blogger blogs to that, then exported to Jekyll and then imported to Hugo.
I exported my WordPress blogs directly to Jekyll and imported to Hugo.
And then the pain started.
I Should Probably Have Automated This Part
I then had to check every post for mistakes during the various migrations.
Sometimes it was:
unconverted- HTML for iframes of SlideShare and YouTube messed up.
- Images were not converted to markdown properly.
- HTML was not converted properly due to a missing tag somewhere.
- Various special chars not converted properly.
Pretty much every post needed to be hand checked and amended with find and replace in some form or another.
It took me ages. I suspect it would have been faster for me to automate this last part.
I Did Automate Something
I dug out my old HTTP Testing Web Crawler and amended it to check for characters on the page, which made it easy for me to check for:
- images that hadn’t been converted properly, e.g. and left

listed on the page rather than a link.
I started hand converting it from a Twine archive into markdown because I looked around and could not find any tools available that did this, and I saw a few failed attempts and I was about to say ‘this is hard.’
But I put together a quick converter using Java and JSoup and it didn’t take long at all.
The benefit is, that because I have automated:
- I am more likely to do it again.
- I will probably write more CYOA games.
- I learned more about JSoup.
- I have yet more code on GitHub.
- I re-used the work I put into pandocifier.
Summary
- I’ve created an archive for my main blogs over at testerhq.com.
- This has an RSS feed that covers - SeleniumSimplified.com, EvilTester.com, JavaForTesters.com, and CompendiumDev.co.uk, and LinkedIn.
- I will continue to add the posts to the other blogs as well, so if you’re only interested in one strand of my work you can keep using that.
- Sometimes we should automate, even if we are only doing a task once.
Published at DZone with permission of Alan Richardson, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments