That Moment Where You Should Have Automated but Didn't

On Risk Aversion

Because I’m slightly risk aware I have the following concerns about my main blogs:

• blogger.com is free so at some point, Google will cancel it and I will lose everything.
• WordPress uses PHP and a database and at some point, someone will take advantage of a security hole and I’ll get hacked and lose everything.

When I post to LinkedIn I have the concern that:

• There is no RSS feed.
• LinkedIn might go down and I’ll lose everything.

And if I ever start posting to medium or other platforms I will have the same issues.

Risk Mitigation

Therefore I decided to ‘archive’ everything into a static site so that if everything goes down I have a backup.

I chose to use Hugo for this.

Subtle Wins

This has the advantage that:

• all posts in Hugo are written in markdown.
• I already write all my posts in markdown (and convert them into HTML for pasting into other platforms).
• I get to use my testerhq.com domain.
• I get to try out the multiple site hosting on my vidahost cloud plan.
• It's much harder to hack a static site (and if you do, I just re-upload).
• I can eventually put all posts on GitHub and have an offsite source backup as well.
• I can repurpose all the posts later.

I will use testerhq.com as my aggregator and I will probably extend it to cover:

• What I post to SlideShare.
• Instagram.
• Tweets I think are useful.
• YouTube videos, etc.

But I Didn’t Automate the Migration

I looked around for tools that could easily migrate from WordPress to Hugo, and Blogger to Hugo but didn’t find a lot that fit my workflow.

I thought I should really write my own, but since I saw so many failed attempts out there I thought it might be too hard and take too long (I really should have done that, though).

I thought that since I’ll just migrate once, that I really couldn’t justify the time to automate it (I might migrate each blog once (4 of them), but there are over 450 posts, that might be worth automating).

I Found a Solution

My solution was:

• Use the WordPress blogger import.
• Use a Jekyll export Wordpress plugin.
• Import from Jekyll to Hugo.

I used a Wordpress Bitnami VM and imported my Blogger blogs to that, then exported to Jekyll and then imported to Hugo.

I exported my WordPress blogs directly to Jekyll and imported to Hugo.

And then the pain started.

I Should Probably Have Automated This Part

I then had to check every post for mistakes during the various migrations.

Sometimes it was:

•   unconverted
• HTML for iframes of SlideShare and YouTube messed up.
• Images were not converted to markdown properly.
• HTML was not converted properly due to a missing tag somewhere.
• Various special chars not converted properly.

Pretty much every post needed to be hand checked and amended with find and replace in some form or another.

It took me ages. I suspect it would have been faster for me to automate this last part.

I Did Automate Something

I dug out my old HTTP Testing Web Crawler and amended it to check for characters on the page, which made it easy for me to check for:

• images that hadn’t been converted properly, e.g. and left ![]( type chars on the HTML.

I still have a lot of broken links to stuff I’ve linked to over the years, and I’ll slowly fix that up over time.

I Almost Repeated the Same Mistake

I was looking at an old Choose Your Own Adventure Game that I created a few years ago in Twine with my son. And thought about converting it into more of a CYOA gamebook format e.g.

listed on the page rather than a link.

I started hand converting it from a Twine archive into markdown because I looked around and could not find any tools available that did this, and I saw a few failed attempts and I was about to say ‘this is hard.’

But I put together a quick converter using Java and JSoup and it didn’t take long at all.

The benefit is, that because I have automated:

• I am more likely to do it again.
• I will probably write more CYOA games.
• I learned more about JSoup.
• I have yet more code on GitHub.
• I re-used the work I put into pandocifier.

Summary

• I’ve created an archive for my main blogs over at testerhq.com.
• This has an RSS feed that covers - SeleniumSimplified.com, EvilTester.com, JavaForTesters.com, and CompendiumDev.co.uk, and LinkedIn.
• I will continue to add the posts to the other blogs as well, so if you’re only interested in one strand of my work you can keep using that.
• Sometimes we should automate, even if we are only doing a task once.