How We Migrated Hundreds of Posts from Hugo to Webflow

We have recently completed the migration of Nebula Graph’s website from Hugo to Webflow, moving from a static website to a more interactive one. There were a number of reasons for this decision, but most notably it was the performance and the ability to do things like form submission and no-code design.

While the redesign of the website on Webflow was easy, since we have an amazing in-house design team and Webflow is also very friendly to designers, the difficult part is migrating more than 100 blog posts we already published from Hugo to Webflow’s Content Management System (CMS).

Basically, these 100 articles were stored as Markdown files on GitHub — not like CMSs like WordPress that stores articles as structured data in a database. In Webflow, the CMS requires structured CSV data to do a bulk import.

Well, everybody thought it would be an easy task before we actually got our hands dirty on the project — Just covert those Markdown files into the HTML format and chuck them into a CSV file, and, bang!

It turned out that things are a lot more complicated, the most notable reason being the limitations of Webflow’s CMS. It only supports a limited number of HTML tags such as <h1> to <h6> and <img> natively. HTML tags like <table> and <code> are not supported if you don’t use its custom code feature — which cannot be used during the bulk import.

In this article, I will introduce how we successfully migrated all contents from our old Hugo website into Webflow. I will also tell you how I solved these tricky problems during the migration and provide some useful scripts. If you are using other static site generators like Hugo and want to migrate your blog site into Webflow, you may find this article helpful.

Some Background

First, a little context about Webflow. Webflow is a visual drag-and-drop website builder that lets you create beautiful, mobile-ready websites without any code. The main benefit I’ve noticed from using it is that I’m able to create visually compelling, responsive websites with very little effort.

In the first version of our website, we chose Hugo, which is a static site generator. We were impressed by its speed and flexibility to do customizations. But as it was a static site, we could not add dynamic content for users who want to comment on our posts or subscribe to our newsletter. Also, it was not easy for the marketing team to upload articles to the website using tools like git.

That’s why we decided to migrate our website to a CMS platform. However, as there are no existing resources about how to migrate from Hugo to Webflow, we had to write our own scripts and manually convert hundreds of posts and pages into collections in Webflow.

Drawing Blueprints for the Article Data

As I mentioned above, our posts were stored as individual Markdown files on GitHub. If we want to convert them into structured CSV files, the first thing we need to do was to create a blueprint for the CSV file.

Here is how our post folder is structured:

----posts

--------random-article-one
------------thumbnail-image-article-one.png
------------index.md

--------random-article-two
------------thumbnail-image-article-one.png
------------index.md

--------random-article-three
------------thumbnail-image-article-one.png
------------index.md
...

We use the name of the folder where the article is stored as the URL slug of the article on the Hugo website. For example, the path to article one will be /posts/random-article-one. There is a slug property in the Webflow CSM which determines the path to the dynamic resource. So we can use the folder name mentioned here as the slug field in the Webflow CMS.

Now let’s look at how each Markdown file is structured. In the file tree above, the [index.md](http://index.md) file under each folder stores all information about a single article.

---
title: "How I cracked Chinese Wordle using knowledge graph"
date: 2022-04-15
description: "Wordle is going viral these days. So is Handle, the Chinese Wordle. This post explains how to crack Chinese Wordle using knowledge graph."
tags: ["use-cases"]
author: "Wey"
---

![How I cracked Chinese Wordle using knowledge graph](https://user-images.githubusercontent.com/57335825/163546889-dff1d1d6-a795-4d88-aff1-9c2b17eacb6e.jpg)

Wordle is going viral these days on social media. The [game](https://www.nytimes.com/games/wordle/index.html) made by *Josh Wardle* allows players to try six times to guess a five-letter word, with feedback given for each guess in the form of colored tiles indicating when letters match or occupy the correct position.

We have seen many Wordle variants for languages that use the Latin script, such as the [Spanish Wordle](https://wordle.danielfrg.com/), [French Wordle](https://wordle.louan.me/), and [German Wordle](https://wordle.at/). However, for non-alphabetic languages like Chinese, a simple adaptation of the English Wordle’s rules just won’t work.
...

The first few lines of this file wrap up all the information of this article. In the first table, you can get the title, date, description, tags, and author of the article. Everything else below the table is the post body.

Please note that while we store the post body in the Markdown format, Webflow’s CMS stores the body in the HTML format. So there must be a conversion between Markdown and HTML before we generate the CSV file.

BTW, actually, we need two CSV files here since tags are stored as a multi-reference field in Webflow. They are posts.csv and tags.csv.

Now we have the blueprint of the post CSV.

Note: You have to create a Post collection in Webflow’s CMS and create fields in accordance with the CSV headers listed above using the exact field names. I mean, you don’t have to use the exact names, but if you do, you won’t have to match fields manually while importing.

Then there is the tags.csv.

Converting Markdown Files to CSV

After the blueprints are done, we can start converting posts in the posts folder into one CSV file. Here is the workflow we used:

1. Iterate every post folder under the posts folder.
2. Use the package gray-matter to extract information from each markdown post file.
3. In the post body field, use showdown to convert Markdown to HTML.
4. Use the package CSV-writer to write information into a CSV file.

Here is the code for the whole process. You can also visit the GitHub Gist of it to make contributions.

const fs = require('fs');
const showdown  = require('showdown');
const matter = require('gray-matter');
const createCsvWriter = require('csv-writer').createArrayCsvWriter;

const csvWriter = createCsvWriter({
    header: ['Title','Slug','Published on','Post Body','Description','Thumbnail image','Author','tags'],
    path: './post.csv'
});

var getAHref = function(htmlstr){
    var reg = /<img [^>]*src=['"]([^'"]+)[^>]*>/gi
    var arr = [];
    while(tem=reg.exec(htmlstr)){
        arr.push(tem[1]);
    }
    return arr;
}

const rtfToMarkdown =(txt)=> {
    // txt = txt.replace(/<p>/g, '');
    // txt = txt.replace(/<\/p>/g, '\n\n');
    txt = txt.replace(/\n/g, '<br/>');
    
    txt = txt.replace(/ /g, ' ');
    
    // console.log(txt);
    return txt;
  }

const download=()=>{
    const files = fs.readdirSync('./posts');
    const csvItems = []
    const tags = new Set([])
    files.forEach(filename =>{
        if(filename !=='.DS_Store'){
            let readMe = fs.readFileSync(`./posts/${filename}/index.md`, 'utf8');
            //Extract header information from the Markdown file
            const _data = matter(readMe);
            const { data, content} = _data
            // console.log("_data", rtfToMarkdown(content))

            //Extract post body from the Markdown file
            const converter = new showdown.Converter({
                noHeaderId: true,
                headerLevelStart: 2,
                literalMidWordUnderscores: true
            })
            let html = converter.makeHtml(content);
            // html = html.replace(/<pre><code>/g, '<h6>');
            html = html.replace(/ /g, ' ');
            html = html.replace(/<pre(([\s\S])*?)<\/pre>/g, item=>{
                item= item.replace(/\s+(?:class|className)=(?:["']\W+\s*(?:\w+)\()?["']([^'"]+)['"]+>/g, calss=>{
                    const _calss = calss.substring(1,calss.length-1)
                    return `>${_calss}`
                })
                item = item.replace(/<pre><code>/g, '<pre><code><h6>');
                item = item.replace(/<\/code><\/pre>/g, '</h6></code></pre>');
                return item 
            });
            data.tags.map(_tag => tags.add(_tag))
            let csvItem = [
                data.title,
                filename,
                data.date,
                html,
                data.description,
                getAHref(html)[0],
                data.author,
                data.tags.join(';')
            ]
            
            csvItems.push(csvItem)
        }
    })
    console.log("tags",tags) 
    csvWriter.writeRecords(csvItems).then(() => console.log('Done~'));
}
download();

Importing the CSV file to Webflow

Now that we have a CSV file of around 110 posts, we thought it would be easy to just chuck it into the Webflow CMS.

But that’s where all the trouble starts.

Challenge No.1: Not All HTML Tags Are Supported by Webflow

Yes, we successfully imported all the posts into the Webflow CMS, and the process was kind of smooth. But it turned out that the posts were not rendered as we expected.

The problem is that Webflow only supports a limited number of HTML tags natively. As a tech startup, our blog consists of lots of articles that contain code blocks. But Webflow doesn’t support the <code> and <pre> tags, which are usually used in combination with tag code blocks.

Natively, Webflow only supports <H1> to <H6>, <p>, <a>, <img>, <ul>, <ol>, <li>, <label>, <strong>, <em>, <blockquote>, and <figure>, according to its documentation. I emphasized “natively” because you can actually write other HTML tags in the custom code block in the CMS. But there is no way that you can upload or wrap your unsupported HTML tags in a custom code block during the bulk import.

Because of the limitations on HTML tags, Webflow will automatically trim all unsupported tags from the body. So say we have a code block that originally look like this:

<pre class="language-python"><code class="language-python">
from nebula3.gclient.net import ConnectionPool
from nebula3.Config import Config

# define a config
config = Config()
config.max_connection_pool_size = 10
# init connection pool
connection_pool = ConnectionPool()
# if the given servers are ok, return true, else return false
ok = connection_pool.init([('127.0.0.1', 9669)], config)
</code></pre>

The post body will only show the code block as plain text without any of the <pre> and <code> tags.

Then we thought of a solution, we can pick a supported tag that is not frequently used in normal blog posts, say <h6>, and substitute all of the <pre> and <code> tags with <h6> tags in the source file, so that the code blocks are wrapped in something during the import. And in the frontend page, we can use JavaScript to change <h6> tags back to <pre> and <code>.

By the way, we use Prism.js to do syntax highlighting for code blocks. We need to use a class in the <pre> and <code> tags to tell Prism which language each code block is written in. For the example above, the class is “language-python”.

And you guessed it right, Webflow will also trim all classes from HTML tags during the import. In order to fix that, we extracted the classes of each <pre> and <code> tag and put them right at the beginning of the <h6> tag.

So for the code block example above, the transformed code will be:

<h6>class="language-python"
from nebula3.gclient.net import ConnectionPool
from nebula3.Config import Config

# define a config
config = Config()
config.max_connection_pool_size = 10
# init connection pool
connection_pool = ConnectionPool()
# if the given servers are ok, return true, else return false
ok = connection_pool.init([('127.0.0.1', 9669)], config)
</h6>

And in the frontend page, we use the following JavaScript to change <h6> tags back to <pre> and <code> and add their classes back. (This script requires JQuery.)

$(document).ready(function() { 
    $('h6').each(function(_,item){
        let newEl = document.createElement("pre");
        let codeEl =document.createElement('code')
        newEl.appendChild(codeEl);
        codeEl.innerHTML = item.innerHTML.replace(/<br>/g, '\n');
        item.parentNode.replaceChild(newEl, item);
    });
    //Add 'language-base' if the class is not specified.
  $('pre').addClass('language-base');
  $('code').addClass('language-base');  
  let _snips = $('code:contains("class=")');
  _snips.toArray().forEach(el =>{
    const txt =el.innerHTML;
    const classStr  =txt.match(/(?:class|className)=(?:["']\W+\s*(?:\w+)\()?["']([^'"]+)['"]/)
    el.setAttribute("class",classStr[1]);
    el.parentNode.setAttribute("class",classStr[1]);
    el.innerHTML =el.innerHTML.replace(/(?:class|className)=(?:["']\W+\s*(?:\w+)\()?["']([^'"]+)['"]/g, '');
    el.innerHTML =el.innerHTML.replace(/<br>/g, '\n');
  });
  var script = document.createElement("script");
  script.type = "text/javascript";
  script.src = "https://web-cdn.nebula-graph.io/nebula-website/js/prism.js";
  document.body.appendChild(script);
 })

Challenge No.2: Line Breaks and Spaces Are Removed during the Import

This should work…except for one thing: Because we used <h6>, Webflow will remove all line breaks and extra spaces from code wrapped inside <h6>. So what we eventually got was something like this:

<pre class="language-python"><code class="language-python">
from nebula3.gclient.net import ConnectionPoolfrom nebula3.Config import Config# define a configconfig = Config() config.max_connection_pool_size = 10# init connection poolconnection_pool = ConnectionPool()# if the given servers are ok, return true, else return falseok = connection_pool.init([('127.0.0.1', 9669)], config)
</code></pre>

Yes, it was one single line of code and all line breaks and extra spaces were committed during the import.

This problem took us quite a few days to solve. And it was in vain.

First, we thought we can use first convert all line breaks and spaces into some placeholders like [linebreak] and [space] and convert them back to real line breaks and spaces in the front-end.

We soon deprecated this idea because this will be a disaster for SEO—While code blocks may appear to be normal to human eyes after the second conversion, what Google’s crawler will fetch from the page will be a bunch of [linebreak] and [space].

Moving on from this idea, we had to turn to Webflow’s technical support for help. And they give us one solution that finally worked.

Here is Webflow’s solution. In the original file, we can add another two <pre> and <code> tags around the <h6> so that, during the import, even though Webflow would trim the <pre> and <code> tags, it would keep all the line breaks and spaces inside <h6>. I guess those <pre> and <code> tags are treated as some sort of sacrifice.

So the original HTML code we need to prepare for the post body will be like this.

<pre><code><h6>class="language-python"
from nebula3.gclient.net import ConnectionPool
from nebula3.Config import Config

# define a config
config = Config()
config.max_connection_pool_size = 10
# init connection pool
connection_pool = ConnectionPool()
# if the given servers are ok, return true, else return false
ok = connection_pool.init([('127.0.0.1', 9669)], config)
</h6></code></pre>

And finally, this worked perfectly. We have all the syntax highlighting for code blocks—as you can see from this page!

One More Thing

Please remember that the <table> tag, and <tr>, <th>, and <td> tags that come with is, are not supported by Webflow as well. For us, we didn’t try to find a universal solution for tables since we don’t use them much. Our solution to this was to manually find all tables in the imported posts in the CMS and substitute them with custom code that contains the table elements.

Conclusion

Well, Webflow is a great platform for designers and marketers. And if you don’t have to worry about migrating posts from other platforms, you are usually fine. But if you are like us and need to migrate hundreds of posts that contain HTML tags that are not supported by Webflow, you may need some extra work. But the good thing is that Webflow allows you to use a lot of workarounds to solve these problems. You can always hack your way out.