VoltDB: Creating an Openshift Cartridge, Part 1

This post was written by Jeff McCormick on March 25th and originally published here.

Introduction

This blog describes a new Openshift cartridge developed for the VoltDB database (www.voltdb.com). I’ve been doing research on VoltDB and figured that others might benefit from having quick access to an Openshift hosted instance of the database. VoltDB is truly impressive in both performance and in capabilities.

A ‘cartridge’ is a means of packaging and deploying an application, in this case the VoltDB database, onto the Openshift PaaS framework.

The VoltDB cartridge in beta form is located at:

https://github.com/jmccormick2001/voltdb-cart

The combination of Openshift PaaS and VoltDB make for some interesting project ideas!

Performance Note

Openshift runs either on the public cloud hosted by Redhat at http://openshift.redhat.com, or you can install Openshift on your own hardware.

Obviously for the highest performance, running VoltDB on your own private infrastructure will provide the highest performance. But for simply testing out VoltDB and learning its various features, the public cloud infrastructure will work fine.

The cartridge we have developed for VoltDB will operate on either the public cloud Openshift or the private versions of Openshift, so the choice is yours.

Installation Scenario

For this cartridge, at this beta stage, I’ve used the VoltDB-provided VoltCache example as the database schema and compiled application catalog which will be installed by the cartridge. Later blogs will show you how to swap out this default application catalog with your own VoltDB application catalog.

Sample Deployment

Create a public Redhat Openshift account at http://openshift.redhat.com.
Log into your account, define your domain name (e.g. voltdbtest):

upload your public key you want to use:

Click on the Add Application button to get started:

Create a DIY (do-it-yourself) Application, the DIY application is at the bottom of the Application.

Next, add the VoltDB cartridge to your application, click on the link that says Continue to the application overview page.:

Next, specify the following URL within the Install your own cartridge section of the web page:

https://raw.githubusercontent.com/jmccormick2001/voltdb-cart/master/metadata/manifest.yml

At this point, if you click on the Add Cartridge button, it will install the VoltDB cartridge into your Openshift application:

At this point, VoltDB should be running on your application host and is ready to test!

Install Openshift Client Utilities

On your client host, in this example a Fedora 20 desktop, you will run the client for the VoltDB VoltCache example. In order to do this, you will need to create networking tunnels to your Openshift application instance. For this, you will need to install the Redhat Openshift client tools called ‘rhc’.

The Openshift client installation is documented at the following location:

https://www.openshift.com/developers/rhc-client-tools-install

Test the rhc client tools by displaying information (using the rhc app show command) about your newly installed VoltDB application:

Also at this point, you can log into your new application by issuing the rhc ssh command as follows:

Start Port Forwarding

After the rhc installation, you will need to create the necessary port forwarding tunnels to your Openshift application instance as follows using the rhc port-forward command:

After the port forwarding has been started, you can now access from your local client (127.0.0.1) the various VoltDB ports that are running remotely on the Openshift cloud application you have installed.

Running the VoltCache Client

At this point, you will need to have downloaded the VoltDB community distribution to your client host. The cartridge is based upon the VoltDB 4.0.2.3 Community Edition. The VoltDB product can be downloaded from:

http://voltdb.com/download/software

The connectivity (at least from my home office) to the public Openshift is really tested by this example and I found that I needed to reduce the speed at which the VoltDB client sends work to the cloud hosted VoltDB instance, so I adjusted the examples/voltcache/run.sh script used to execute the client as follows:

function benchmark() {
srccompile
java -classpath obj:$APPCLASSPATH:obj -Dlog4j.configuration=file://$LOG4J \
voltcache.Benchmark \
–threads=10 \
–displayinterval=5 \
–duration=20 \
–servers=localhost \
–poolsize=1000 \
–preload=true \
–getputratio=0.90 \
–keysize=32 \
–minvaluesize=1024 \
–maxvaluesize=1024 \
–usecompression=false

Here is an example of running the VoltDB VoltCache example client which performs a benchmark:

VoltDB Studio Web Application

A nice utility that is shipped with VoltDB is their Studio web application, found at the following URL:

http://127.0.0.1:16000/studio/

Using Studio, you can issue a SQL query against the deployed VoltDB application. Notice that the URL here is based on the rhc port-forward command being issued.

Cartridge Development

Getting the cartridge to work in the Origin version of Openshift was a bit easier because you have the ability to bind to localhost (127.0.0.1). In the public Openshift, binding to localhost (127.0.0.1) is not allowed.

To get VoltDB to work in the public version of Openshift, there were a few steps required including:

adjusting ports

In the public Openshift, there are various port conflicts and protected ports, to avoid these problems, I’ve adjusted the VoltDB ports as follows:

1. zookeeper port is 16300
2. internal port is 16700
3. http port is 16000

creating http host command line parameter

In this version of VoltDB, the internal HTTP server was binding to localhost by default and there was no way to specify a different host, so I modified the VoltDB code to allow for a new command line parameter called httphost to let me use the Openshift assigned internal IP address (e.g. OPENSHIFT_DIY_IP)

creating zookeeper host command line parameter

Also, in this version of VoltDB, the internal Zookeeper host was defined as localhost, so I created a new command line parameter called zkhost which allows me to override localhost with the Openshift internal IP address (e.g. OPENSHIFT_DIY_IP).

altering run.sh

To start VoltDB, I need to pass in various parameters to the voltdb command as follows:

voltdb create –internal 16700 –externalinterface $OPENSHIFT_DIY_IP –zookeeper 16300 –httphost $OPENSHIFT_DIY_IP -d $OPENSHIFT_VOLTDB_DIR/sample/deployment.xml –zkhost $OPENSHIFT_DIY_IP -H $OPENSHIFT_DIY_IP –internalinterface $OPENSHIFT_DIY_IP

To stop the VoltDB, I need to pass in the new server host IP address to the voltadmin command as follows:

voltadmin pause -H $OPENSHIFT_DIY_IP;
voltadmin shutdown -H $OPENSHIFT_DIY_IP;

bundling and installing 64 bit JRE

VoltDB requires a 64 bit JRE to run. Interestingly, the public Openshift does not include a 64 bit JRE but rather a 32 bit JRE even though it runs on 64 bit RHEL 6.5! So, I needed to bundle the 64 bit JRE within the cartridge in order to have a proper JRE for VoltDB to run with.

Next Steps

This initial blog shows you some very basic ways to deploy a VoltDB database to the Openshift PaaS. In upcoming blogs, I’ll show you how to modify the deployed VoltDB database schema / compiled catalog and also how to deploy it to Openshift web frameworks other than the DIY framework.