Easily Build and Run Unikernels With OPS
Unikernels can be a great help when building out microservices. In this article, a developer explains the open source unikernel project he's working on.
Join the DZone community and get the full member experience.
Join For FreeUnikernels have long been touted as the next generation of cloud infrastructure for their size, security, and performance — aspects that lend themselves to megatrends like microservices and serverless while combating the non-stop onslaught of data breaches, cryptojacking, and other problems.
There's a big problem though. There has been very little developer pickup. Some of those outside the ecosystem have pointed their fingers at fake problems like a lack of debugging but that simply isn't the reason. For instance, everyone I know that uses unikernels heavily utilizes gdb.
Having been in the space for a few years, there are two big reasons that I can easily point to.
One, most of the public clouds are today not great fits for unikernels because they were inherently built to run monolithic Linux VMs. We are starting to see that change, though, with initiatives like gVisor from Google and Firecracker from AWS.
Two, building the damn things turns out to be difficult for the majority of potential users. If you are a heavy node user or a Pythonista or a Gopher, chances are you don't relish having to twiddle linker or cflags in a large makefile. Chances are you are not used to patching MySQL with custom code. Chances are you are not used to using a different C standard library.
In my opinion, you shouldn't have to be either. For unikernels to take off, they need to be so easy to use that non-developers can consume them.
This is what we are looking at achieving with a new tool we have open sourced recently called OPS. OPS allows anyone, including non-developers, to build and run unikernels easily, instantly, on their own laptop or on a server in the cloud with no prior experience, no signup, no coding, and with a single command. You don't need to re-write any of your code and you can use off the shelf software.
Ok, this is DZone, so let's jump into the code and build and run your first unikernel. Start by downloading the OPS tool. You can run this on Mac OS or Linux. It's a simple Go wrapper around QEMU — probably the most heavily used open source platform for running VMs out there.
curl https://ops.city/get.sh -sSfL | shIf downloading raw binaries this way gets to you it's also possible to build the source as well. That can be found here: https://github.com/nanovms/ops.
We can start off with a small Node.js hello world to get our feet wet. Put this into the hi.js file:
console.log("Hello World!");From there we can load a Node package and run it:
$ ops load node_v11.15.0 -a hi.jsWhat this does is download a Node package that has everything that is necessary to run Node programs. There are many other packages as well that you can find via:
$ ops listHowever, OPS was built to run raw ELF binaries as you would find them on Linux. So let's go ahead and try something a little bit more advanced.
package main
import (
"log"
"net/http"
)
func main() {
fs := http.FileServer(http.Dir("static"))
http.Handle("/", fs)
log.Println("Listening...on 8080")
http.ListenAndServe(":8080", nil)
}This will implement a small Go webserver that can serve static files. If you are on a Mac, you'll need to specify the cross-compilation target of Linux to get an elf, but if you're on Linux no worries.
$ GOOS=linux go build main.goNow let's create a static folder to put some stuff in:
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>A static page</title>
</head>
<body>
<h1>Hello from a static page</h1>
</body>
</html>This time we won't be using a package (since Go is a compiled language not an interpreted one).
{
"Dirs" : ["static"]
}What we've done here is told OPS that when it builds a VM for us to go ahead and place the folder 'static' into the resulting filesystem along with anything else we want/need for this to run. There are actually quite a few options that you can put into this config.json but we've kept it simple for this example.
$ ops run -p 8080 -c config.json serverWe'll specify the port, 8080 in this case, pass it the JSON configuration, and run our server.
$ curl http://127.0.0.1:8080/hello.htmlNow you should be able to hit it with curl and get back your response. It's important to note that by default OPS will resort to using 'usermode' networking and not enable hardware acceleration. Those are both available and would be required for production use, as without them it's too slow, but it's fine if you want to play around in a dev/test environment.
It's also important to note that there isn't actually Linux underneath this image you built. Take a look inside your working directory and check out the size of the image that is created. For me, Node is much larger but that's only because the binary itself is large. It's important to note that unikernels aren't optimizing anything (at least not yet) but they don't need a full fleged general purpose operating system either.
➜ t ls -lh
total 91608
-rw-r--r-- 1 eyberg staff 19B Feb 10 15:40 hi.js
-rw-r--r-- 1 eyberg staff 45M Feb 10 15:41 imageIf you built the Go example you'll see it's much smaller; but what if you wanted to build a C hello world? Well, without switching out libc or anything else we get this:
eyberg@s1:~/c$ ops run main
Downloading.. .staging/mkfs
271.91 KiB / 271.91 KiB [===========================================================================================================================================================] 100.00% 6.59 MiB/s 0s
Downloading.. .staging/boot.img
22.50 KiB / 22.50 KiB [============================================================================================================================================================] 100.00% 35.28 MiB/s 0s
Downloading.. .staging/stage3.img
1.40 MiB / 1.40 MiB [==============================================================================================================================================================] 100.00% 34.17 MiB/s 0s
Finding dependent shared libs
booting image ...
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
assigned: 10.0.2.15
yoyoyoy
exit_groupexit status 1
eyberg@s1:~/c$ ls -lh
total 3.9M
-rw-rw-r-- 1 eyberg eyberg 3.9M Feb 10 23:44 image
-rwxrwxr-x 1 eyberg eyberg 8.4K Feb 10 23:44 main
-rw-rw-r-- 1 eyberg eyberg 70 Feb 10 23:44 main.c#include <stdio.h>
int main() {
printf("yoyoyoy\n");
return 0;
}It's clearly early days for OPS and we have quite a few un-announced surprises coming down the pipe in the next few months but we hope that it allows people to start using these today and start figuring out all the other software that should be built in the ecosystem. I'm sure if you are reading this article you've seen the CNCF ecosystem graphic. The unikernel one is going to absolutely dwarf that namely cause it enables so much newer software to be built but also cause it allows the existing virtualization ecosystem (which is massive) to flourish more and that ecosystem has been around for 20 years now.
OPS was built with modularity in design because unikernels as an infrastructure pattern are so much bigger than what some people have made them out to be. The NFV usecase is different from the edge case which is different from the cloud case which is different from... well you get the picture. Unikernels enable so many new types of compute to be explored and built. Unikernels allow you to do things that simply are not achieveable with normal Linux VMs or containers today. What happens if you build a new serverless runtime on these? What happens when you build a new edge solution out on them? What happens when you realize you can spin up thousands of these in milliseconds and spin them back down again instantly? These are just a few of the interesting applications researchers from the largest software companies out there are using them for today — it's no wonder that IBM lists it's number one cloud patent from last year as a unikernel-based hypervsior or that even Red Hat is working on unikernels supervised by none other than Ulrich Drepper. I was even at a unikernel conference in Beijing last year with developers from Baidu, Alibaba, ARM, etc.
So goto https://github.com/nanovms/ops. Fork it, star it, clone it, whatever, but please let me know what you end up building with it.
Opinions expressed by DZone contributors are their own.
Comments