Let's face it, (IoT) cloud services can end up being very expensive. Although the providers tout one pay only for the computing and network resources used, in the end, one still ends up paying dearly for these services. The cloud service providers also tout virtually 100% uptime, however, the latest Amazon fiasco tells us a different story.
In this article, we look into using alternative solutions that can act as a cloud server by using multiple Virtual Private Servers (VPS) in combination with Round Robin DNS. We also show you some issues you may run into when designing IoT solutions and how to solve them. We round up the article by showing an IoT thermostat concept we designed, where multiple users can simultaneously control the same thermostat in real time.
Round Robin DNS
The typical cloud server provider makes it possible to have a pool of servers all look as if they originate from the same IP address. Instead of having a pool of servers appear as they originate from the same IP address, we can use a pool of servers, each with a unique public IP address, appear as one by using Round Robin DNS.
Round Robin DNS Load Distribution
Round Robin DNS is simply DNS configured to use multiple IP addresses. The DNS server will automatically rotate the list of IP addresses when queried. A quick test on how this works is to resolve the IP address for the domain name simplemq.com. This domain name maps to four IP addresses (four servers). You can ping the domain name from the command line, but ping will only give you the first IP address in this list. To see all IP addresses, use an online service such as MX Toolbox.
The four IP addresses resolved above are for four Virtual Private Servers running in four different datacenters in the U.S., in different geographical regions. By running the servers in different datacenters, we avoid the single point of failure in the datacenter's network switches. From my experience, switches and load balancers are single points of failure that typically break much faster than the servers.
How to Configure Round Robin DNS
Your domain registrar's included DNS service supports Round Robin DNS. Here's how you can setup your own Round Robin DNS test for learning purposes. Use the free domain registrar freenom and register a free domain name. Go to the DNS management page for the domain name and enter the IP address of two or more servers. Leave the name field blank. The following screenshot shows my setup.
The DNS should map to servers with public IP addresses, but for test purposes, you can use private IPs as I have done above. I have two web servers running on my own private network; two machines with IP 192.168.1.100 and 192.168.1.111. For your test, you may alternatively set the DNS to the IP address of any two public servers if you cannot easily start two web servers on two of your computers on your local network.
After configuring your own test domain using freenom, try pinging the domain name from the command line. When I ping my domain name from the command line, I see the following.
Note that since DNS is cached by the browser, your operating system, your ISP provider, and so on. It may take some time before you see the IP address change. Each time I ping the domain name, I see the same IP address since the DNS is cached. Eventually, the DNS Time To Live (TTL) expires and the other IP address is shown when I ping the domain name.
Round Robin DNS Redundancy
When a network client such as a browser resolves a domain name, the DNS server returns not one, but all IP addresses to the network client. A properly behaving network client will try the IP addresses returned in the order listed, one after the other, until the client can connect. If the first server is down, the next one will be tried, and so on. You can easily test this by using the above DNS setup and having two (local) web servers running. Ping the domain name to get the first IP address in the list. Stop the web server associated with this IP address. Now, use a browser and navigate to the domain. After some time, when the browser is unable to connect to the first web server, the browser will connect to the second web server. You will notice a slight delay in the browser before the web page for the second web server is displayed.
Low-Cost IoT Cloud Server Solution Based on Round Robin DNS
A low-cost IoT cloud solution using Round Robin DNS is based on one to N number of low-cost Virtual Private Servers. Just because the servers are low cost does not mean they are low quality. The following is the printout from running the command "uptime" in one of the low-cost servers I am running.
$ uptime 13:31:55 up 675 days, 3:49, 1 user, load average: 0.06, 0.04, 0.05
What the above command tells us is that the server has been running 24/7 for almost two years. How to find a low-cost Virtual Private Server and how to remotely access the server is explained in the DZone article Setting Up Your Own Arduino IoT Server.
Clustering (Merging the Cloud Server Nodes)
If your cloud solution can use server nodes operating in complete isolation, read no further. For most IoT solutions, such a setup is not feasible, especially in Industrial IoT (IIoT). As an example, in IIoT ,it is common for an operator to control many devices from an operator terminal such as by using a Human Machine Interface (HMI). In an IIoT solution, the devices and the HMI connect remotely to the cloud server solution. An IoT protocol running on the cloud solution may be in charge of routing the messages between the HMI and the devices.
Now the problem, in a cloud solution the devices may not be connected to the same server as the HMI. Say we have a cloud server solution based on four Virtual Private Servers as shown in the following figure:
An IoT protocol routing messages between connected network clients is typically referred to as a broker.
What if we have a hundred devices connected evenly to each of the four online server nodes, where each server node is managing 25 devices. An HMI connected to server node one can control the 25 devices connected to this server via the broker, but how can the HMI control the other 75 devices? Note that you have this problem regardless of Cloud Server solution; in other words, the cloud server providers will typically not provide a ready to use solution for this problem.
What we need is a software solution that creates a cluster and that seamlessly and transparently bridges the communication between the cluster nodes, thus making the cluster nodes behave as one physical unit. A good choice would be to select a broker that supports clustering as the base for an IIoT solution. The following figure shows how four brokers are connected together as a cluster, thus making the cloud server nodes behave as if they are one physical server.
The cluster-enabled broker runs as a process on each of the Virtual Private Servers -- i.e. the cluster nodes. The broker is in charge of routing messages between connected IoT clients. Messages are routed between two cluster nodes if the clients exchanging messages are connected to two different cluster nodes.
Bridging a Cluster to Third-Party Services
Many IoT solutions require bridging with third party services. Bridging together services and protocols can be done on the server side with a custom application. Such an application must be replicated across all nodes part of the cluster.
We designed a weather application that showcases various IoT concept, including bridging to third party services such as weather data services. The Weather application consists of a thermostat (device) with a native user interface. This user interface can be synchronized with a browser user interface. The complexity is in bridging web browsers connected to different cluster nodes than what the device is connected to. For the design, we used the Mako Server and its integrated IoT protocol SMQ, which provides clustering and bridging. When a browser is synchronized with a device, but connected to a different cluster node than the device, the SMQ IoT protocol enables the custom server-side application to permanently re-route the messages to the correct server side weather application. The following figure shows two browsers at the top connected to the cluster and synchronized with the server-side weather application on the node where the device is connected. The device user interface is shown at the bottom.
The custom server side application shown as the dark brown rectangle on each cluster node above is implemented in the Lua scripting language and is designed specifically for bridging third party services with the IoT solution and to initiate routing of messages destined for server side weather applications on other nodes. Notice how all the four server-side weather applications interact with third party services. The four servers are identical clones and the software installed on the servers is automatically synchronized by using csync2. If you are interested in testing this application, download the device's simulated weather user interface for Windows. See the Weather App Documentation for details.