DZone
Agile Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Agile Zone > That Moment Where You Should Have Automated but Didn't

That Moment Where You Should Have Automated but Didn't

One of our MVBs shares his experiences with moving his blog to a central archive, and how he quickly realized automating migrations is never a bad idea.

Alan Richardson user avatar by
Alan Richardson
·
Apr. 14, 17 · Agile Zone · Opinion
Like (1)
Save
Tweet
4.20K Views

Join the DZone community and get the full member experience.

Join For Free

On Risk Aversion

Because I’m slightly risk aware I have the following concerns about my main blogs:

  • blogger.com is free so at some point, Google will cancel it and I will lose everything.
  • WordPress uses PHP and a database and at some point, someone will take advantage of a security hole and I’ll get hacked and lose everything.

When I post to LinkedIn I have the concern that:

  • There is no RSS feed.
  • LinkedIn might go down and I’ll lose everything.

And if I ever start posting to medium or other platforms I will have the same issues.

Risk Mitigation

Therefore I decided to ‘archive’ everything into a static site so that if everything goes down I have a backup.

I chose to use Hugo for this.

Subtle Wins

This has the advantage that:

  • all posts in Hugo are written in markdown.
  • I already write all my posts in markdown (and convert them into HTML for pasting into other platforms).
  • I get to use my testerhq.com domain.
  • I get to try out the multiple site hosting on my vidahost cloud plan.
  • It's much harder to hack a static site (and if you do, I just re-upload).
  • I can eventually put all posts on GitHub and have an offsite source backup as well.
  • I can repurpose all the posts later.

I will use testerhq.com as my aggregator and I will probably extend it to cover:

  • What I post to SlideShare.
  • Instagram.
  • Tweets I think are useful.
  • YouTube videos, etc.

But I Didn’t Automate the Migration

I looked around for tools that could easily migrate from WordPress to Hugo, and Blogger to Hugo but didn’t find a lot that fit my workflow.

I thought I should really write my own, but since I saw so many failed attempts out there I thought it might be too hard and take too long (I really should have done that, though).

I thought that since I’ll just migrate once, that I really couldn’t justify the time to automate it (I might migrate each blog once (4 of them), but there are over 450 posts, that might be worth automating).

I Found a Solution

My solution was:

  • Use the WordPress blogger import.
  • Use a Jekyll export Wordpress plugin.
  • Import from Jekyll to Hugo.

I used a Wordpress Bitnami VM and imported my Blogger blogs to that, then exported to Jekyll and then imported to Hugo.

I exported my WordPress blogs directly to Jekyll and imported to Hugo.

And then the pain started.

I Should Probably Have Automated This Part

I then had to check every post for mistakes during the various migrations.

Sometimes it was:

  •   unconverted
  • HTML for iframes of SlideShare and YouTube messed up.
  • Images were not converted to markdown properly.
  • HTML was not converted properly due to a missing tag somewhere.
  • Various special chars not converted properly.

Pretty much every post needed to be hand checked and amended with find and replace in some form or another.

It took me ages. I suspect it would have been faster for me to automate this last part.

I Did Automate Something

I dug out my old HTTP Testing Web Crawler and amended it to check for characters on the page, which made it easy for me to check for:

  • images that hadn’t been converted properly, e.g. and left ![]( type chars on the HTML.

I still have a lot of broken links to stuff I’ve linked to over the years, and I’ll slowly fix that up over time.

I Almost Repeated the Same Mistake

I was looking at an old Choose Your Own Adventure Game that I created a few years ago in Twine with my son. And thought about converting it into more of a CYOA gamebook format e.g.

walk out of the door (goto 8) 

listed on the page rather than a link.

I started hand converting it from a Twine archive into markdown because I looked around and could not find any tools available that did this, and I saw a few failed attempts and I was about to say ‘this is hard.’

But I put together a quick converter using Java and JSoup and it didn’t take long at all.

The benefit is, that because I have automated:

  • I am more likely to do it again.
  • I will probably write more CYOA games.
  • I learned more about JSoup.
  • I have yet more code on GitHub.
  • I re-used the work I put into pandocifier.

Summary

  • I’ve created an archive for my main blogs over at testerhq.com.
  • This has an RSS feed that covers - SeleniumSimplified.com, EvilTester.com, JavaForTesters.com, and CompendiumDev.co.uk, and LinkedIn.
  • I will continue to add the posts to the other blogs as well, so if you’re only interested in one strand of my work you can keep using that.
  • Sometimes we should automate, even if we are only doing a task once.
POST (HTTP) Moment

Published at DZone with permission of Alan Richardson, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Vaadin Apps as Native Executables Using Quarkus Native
  • Modern Application Security Requires Defense in Depth
  • Delegating JWT Validation for Greater Flexibility
  • Blocking Ads on Your Network Using Raspberry Pi 3 + Fedora + Pi-hole

Comments

Agile Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo