Building CMS Backed Augmented Reality Apps With Crafter CMS, Part 1

From Snap Chat to Pokemon Go to the helmets used by F-35 fighter pilots, Augmented Reality (AR) applications are gaining a ton of traction. Unlike virtual reality which attempts to completely replace actual reality with an alternative universe and sensory experience, augmented reality, leverages technology to add a sensory (typically visual) layer of interaction and data on top of our everyday experiences. You can learn more about the difference between VR and AR by watching this free tech-talk / webinar. Augmented reality is so exciting and opens up so many new opportunities because it enhances our physical experience, giving us a way to experience things in a way we never could before.

As with most applications today, AR applications frequently contain a lot of content and related digital media such as 3D models, images, and videos. Often times, textual/markup content and digital media are deployed statically along with the application. This approach creates unwanted dependencies for content updates on the developers and requires laborious deployment processes.

In a business context, this misalignment and lack of independence can slow activities way down and ultimately impact the bottom line. To address this common problem, developers often build custom applications for non-technical users or leverage content management system (CMS) technology. Building a custom authoring tool doubles development time and creates an additional support burden on your development team. Leveraging a CMS is a great idea, but you need to make the right choice. Many CMS architectures impose severe limitations on markup, application, and API architecture and design. As we’ll see in this article, Crafter CMS provides you with the tools and flexibility you need to create innovative, rich experiences like Augmented Reality.

In this article, we’ll build a simple (but not too trivial) AR application. After that, we’ll update it so that non-technical users can upload new models and edit and deploy/publish changes to the scene through Crafter CMS without any technical background.

For our application, we’ll start with a simple AR application based on open source frameworks AR.js, ARToolkit, and THREE.js. Given that this is a tutorial and we want to have some fun with it, we’ll use (USC 17 107) some freely available 3D Models from ROE Studios. You may also use any other models as long as they can be loaded by THREE.js’s library of loaders. Free models vary in availability and quality. I chose this library because of the number of available models. 

Step 1: Get a Simple, Static Browser-Based Augmented Reality Application Running

Before we put anything in the CMS, let’s first make a basic static AR example application work. We won’t get into the nuts and bolts of the various AR and 3D web related libraries. There is a lot of material and examples already available on the subject. Instead, we’ll try to avoid a lot of abstraction and construct an example that is simple enough in what it does so that the code can be readily understood. You can find the code here on my Github account.

• Clone or download this code as a zip.
• Open code/simple-ar.html with a browser.

In your browser, you should see a video of yourself (captured by your camera if it’s available and you have authorized it to be used by the browser). Additionally, hovering on screen, there should be a 3D model of Pikachu.

Now that we have a VERY SIMPLE working example of an AR application, we can get started integrating content management capabilities with our CMS so that anyone can add new 3D models to the application.

Step 2: Get Crafter CMS Downloaded and Installed

The first step is to get the CMS downloaded and installed. Crafter CMS is free and open source software licensed under GPL v3.

2.1 Download the Crafter CMS bundle

• Download Crafter CMS here: http://craftersoftware.com/downloads
• Simply click the Download button.

2.2 Install Crafter CMS and Start It Up

• Unzip the download.
• Executestartup.sh (or startup.bat if you using Windows) inside the unzipped folder.
  • Prerequisite: Java 8 JDK is installed, and JAVA_HOME is defined as an environment variable.
• Open a browser to http://localhost:8080/studio. I recommend using Chrome. Chrome has the best support for desktop file drag and drop of all the browsers which will come into play later in this article.
• Login with username ‘admin‘ and password ‘admin.’

Step 3: Install the Application as a Project in the CMS

We now have the prerequisites done, and we’re ready to party! Let’s install the application into the CMS.

3.1 Create and Setup a CMS Project to Manage Your Application

• In Step 1.2, you logged into the CMS as the admin user who has an admin role. On the screen, you will see a button on the left hand that says “Create Site.” Locate and click this button.
  
  
• Enter AR Tutorial as the application name. Crafter CMS will generate a Site ID for you.
• Select “Empty” for your blueprint and click the Create button. Crafter CMS will create a new project for your AR Tutorial application with a basic “Hello World” type configuration.

3.2 Upload Your Application to the CMS

• Click on Site Content in the menu at the top of the screen. This will open a panel on the left
• Click on the Static Assets label. This will open the folder.
• Right click on the static assets folder and choose Bulk Upload Assets.
  
• In your code folder from step 2, you will find a static-assets folder. Drag the contents of this folder onto the upload dialog in the browser. When it’s done, you will be able to clickDone.
• By uploading the compiled assets, we’ve loaded the app into the CMS. Now we have to tell the CMS to render our single page. To do this, we must update our out-of-the-box template provided to us by the blueprint. In Crafter CMS, templates are coded in Freemarker and can contain any kind of markup (typically HTML 5) as well as Freemarker Directives that make it dynamic.
  

  3.3 Update the Site Template With the Content From the HTML Page

• To update the template click “Site Content” in the menu at the top to hide the Site Content panel. Then locate the Wrench in the upper right-hand corner on the toolbar. Click the Wrench. This will open the Preview Tools panel on the right-hand side of the screen.  

• Now click on In-context Editing to expand that specific tool. You will notice this tool allows you to edit content, templates, and controllers. We’re interested in the templates.  
• Click Edit Template.
  

• Replace the code in this template with the HTML found in your build directory at code/simple-ar.html.Note that the static code example assumes the application assets are at ./static-assets on the server. In Crafter CMS, static assets like CSS, JavaScript, and images are served from /static-assets. You will want to modify the paths (remove the ‘.’) in the HTML template to use the /static-assets folder as shown above. Once done, click update.

Your app is now running in the CMS!  Congrats!  With this complete, we can start working on making the application dynamic, editable by a non-technical user and updatable at any time without a deployment.

Step 4: Content Enable the Application

The first step in content enabling the application is determining what the content model is. A content model is a definition and structure of the content. We maintain the definition of the content structures within Crafter CMS by declaring Content Types.

To figure out what your Content Types should be, look for the NOUNS in the application that you want a non-technical user to be able to easily update and publish at any time. In our case, this is an individual 3D model of a Pokemon like Pikachu.

While the model in this simple application will be very simple, Crafter CMS is capable of modeling very sophisticated objects as well as relationships between objects to handle real world scenarios.

Before getting into the CMS, let’s figure out what the definition for our Pokemon type should be. We do this by looking at the application and seeing what fields the app will require. Because this particular type of content is a 3D asset rather than traditional digital media, we need to understand a little about what goes into making a model and how it’s described. We’ll start by looking at the two models in the model directory. By examining both the Pikachu and Snorlax model directories we can see that a model can contain a different number of files. Complex models may contain a lot of files.

We want the act of adding these models to the CMS to be really easy. Other than adding the files to the CMS there is nothing to configure about them. Further, a folder is a good grouping mechanism for keeping the files for a specific model together. Given this, we’ll assume that the first step for the author in adding a new model is to simply upload it by dragging the model folder into the static-assets model directory via bulk upload. In other words, for the 3D models themselves, we’ll leverage folder structure and IA rather than Content Model to store and organize the raw 3D content assets.

The next thing to look at is how the model is loaded into, positioned, and scaled in the world.  For this, we need to look at the code. Specifically, the code that loads the 3D model:

By examining this we see there are a number of important attributes:

• We also see that every model has a file that defines its structure (.obj).
• And a file that defines how to apply its textures (.mtl).
• The Model Scale Multiplier.
• The Object Rotation on the X axis.
• The Object Position on the X axis.
• The Object Position on the Y axis.

If we think about it, some of these are directly related to the 3D model itself and belong to the Pokemon content model. Other attributes like scale and position are used to place the model in the world. It’s more flexible if we manage the scale and positioning attributes outside of the object.

OK! We have a list of fields we need for our Pokemon, let’s get started defining the Content Type in the CMS:

4.1 Declare a Pokemon 3D Model Content Type

• Open the Site Content panel.
• Locate and click the menu item for Admin Console. This will take you to the power user area in the CMS used to manage and perform various site administration tasks.
• On the left panel in the Admin Console, locate and click the menu option for Content Types. This tool is used to create new Content Types and edit existing ones.

• On the toolbar at the top of your screen, you will notice two options: Open Existing Type and Create New Type. Because a Pokemon is a new concept you want to introduce and define in the CMS, click on Create New Type. This action will pop up a dialog to prompt you for some basic information about the content type.
• Enter Pokemon for the display label. Crafter CMS will suggest a content type name automatically for you.
• The Type you want to choose is Component. In Crafter CMS anything that you are modeling which is NOT a page is a Component. The main difference in the CMS between pages and components is that page objects are automatically URL addressable while components are not.
• Once the dialog is configured, click Create. This will open the content type editor for your Pokemon Content Type. You can learn more about the content type editor in detail at http://docs.craftercms.org/en/stable/developers/content-modeling.html
  

• The Content Type Editor lets you drag/drop editing field types from the right-hand controls list into your Content Type definition. You then click on the individual controls and configure them with Labels, properties, data sources, and constraints.

• For each field, you need to add (Object File, Texture file, etc.), look through available controls list on the right, find the appropriate editing control for that content, and drag/drop it into the Pokemon properties area.
• After you add a field to the Content Type, click on it to configure it in the Properties Explorer.
  • Add a Title. This is the user-facing name the author will see alongside the input control.
  • Crafter CMS will automatically suggest a variable name which you can use later in templates and controllers to access the content values.
  • Scroll down in the Properties Explorer to see all the properties and constraints for the given field. This is how you mark fields as required, change their input length, and make other adjustments.
    

• Because you want the user to be able to pick the object and texture fields, you will need to give them a file picker control. In Crafter CMS, this is called the Item Selector control. Drag one of these on the screen as shown above and give it a title of “Object File.”

• Now you need to tell the Item Selector where to find the content it’s going to allow the user to pick from (remember the authors will upload models into the /static-assets/model folder via drag and drop). To wire this up, you will need a Data Source. In Crafter CMS, the way picker type controls get their values is via a Data Source. There are several out of the box Data Sources, and it’s possible to define your own as well. For our example, because we want to pick an existing file in the repo, we’re going to use the data source called File Browse. Scroll down within the Data Sources area until you see it, then drag/drop it onto the Data Sources portion of the Content Type definition.
  Click on added Data Source and begin to configure it as shown above. Note that we’re simply identifying a path elsewhere in the CMS where we want to pick a file from.
• Once our Model Files Data Source is defined, let’s connect it to our Item Selector control for Object Files. Click on the Object Files control and configure it in the property editor as shown below:
  
• Add another field for Text Files and connect it to the same data source.
• Add an input control for X Rotation and configure it as shown below:
• Perfect! Your Content Type definition is now complete. Click Save at the bottom of the screen. Note that you are going to get a warning that the content type does not have a template associated with it. That’s ok. Not every Content Type needs a template. In our case, the app already knows how to render the content, and we don’t need another template. Click Save on the dialog. You’ll notice a small save indicator upon successful save.

4.2 Create Pokemon Content Objects

Now it’s time to create some test content.  To return to your project Click the Crafter Logo on the toolbar in the top left corner of the screen. This takes you to your workflow dashboard. From here you can create content or navigate back to your application preview via the Site Content panel.

• Click on Site Content in the menu at the top of the screen. This will open a panel on the left.
• Click on the Components label. This will open the folder.
• Right click on the components folder and Create Folder.
  

• Provide the folder name “pokemon”.
  

• By clicking Create, you will have added a Pokemon folder to the components folder. This is simply to help organize your content.
• Right click on the Pokemon folder and Create Content.

• The CMS will prompt you for the type of content you want to create. You will note the Pokemon option. Choose Pokemon and click “Ok.”

• Once you click OK, you will be given the entry form for your Pokemon which you can start filling out. Awesome! Crafter CMS automatically creates your Pokemon editing interface based on the Content Type Definition.
  

• Repeat this process as many times as you want, creating a new Pokemon each time. When you save the form, you will see the Pokemon item listed under the Pokemon folder as whatever value you entered into the Internal Name field (which is only used by the CMS editing tools as a label in the tool).

• Note: I included an X Access Rotation in the model object because, based on the available models, the models all start facing in different positions. The same is true for scale. You may wish to add a “normalizing” scale value in addition to the rotation.

Tune in tomorrow for the second part of this article!