Sending IoT Data From Arduino to Questdb
Sending IoT Data From Arduino to Questdb
How to send IoT data from Arduino to Questdb in a few simple steps.
Join the DZone community and get the full member experience.Join For Free
IoT on QuestDB
As you can well imagine, I've been super busy in my new job at QuestDB so this has taken me longer than I would have liked, but here we are. If you know me at all, you know that one of the first things I always do with new things is, well, connect things to them! So I immediately went to connect an IoT device to QuestDB to see how it went. Unsurprisingly, it went quite well. So here's how it went, from start to finish.
The Database Part
The first thing I had to do was to get QuestDB up and running. Luckily, this is very straightforward. I guess I could have gone the Docker route, but as you're probably aware, I'm not a huge fan of Docker (in no small part due to the fact that it will literally suck the life out of a macOS laptop). There's also (for you MacOS users) `brew install questdb` but since I work here, and I wanted to test out the latest and greatest web console, I decided to build from source:
It builds really quickly due to the lack of external dependencies, so that is great! Then all I have to do is start it:
That is literally all there is to getting QuestDB built and running. But that's just the first part. Now it's time to do something mildly useful with it. First, I'll need to create a table in QuestDB to store my IoT Data (A bit more on this later, so store a pointer to this).
Remember, we're doing SQL here, so there's no new language or syntax to learn. This is a really simple table that I'm building because I'm going to be using an ESP8266 with a (really awful) DHT11 temperature and humidity sensor on it.
The Sensor Part
For this, I'm going to use an ESP8266-based WEMOS D1 Mini only because I happen to have a giant pile of them lying around. I buy them in bulk because they are a dollar or 2 each, easy to use, and largely disposable if I blow one up (which I do with alarming regularity.). The circuit is extremely simple to do:
I used an actual WEMOS Shield with the DHT11 on it, so I didn't have to breadboard it, but this schematic gives you an idea of how simple the wiring is. It's literally 3 wires.</p>
The Code Part
Here is where the magic happens. How I actually send the sensor data to the database. There is a simple example program included with the Adafruit DHT Unified Sensor Library that I recommend starting with in order to make this a bit easier. It already has all the parts to read from the sensor so you don't have to write those from scratch. Remember: Good developers copy, but great developers paste!</p>
Since I'm using the 8266, and I'll need internet connectivity, I'll need all the WiFi bits:
Really all you have to do is go to the 'Sketch' Menu, Choose 'Include Library' and select the 'ESP8266WiFi' library and you get all this stuff imported for you.
Here's some boiler-plate code you can always use to get your ESP8266 on your WiFi:
That gets the basics set up. Running that should get you a WiFi connection and a fully configured DHT11 sensor. We're almost ready to start sending data to the database
If you were paying attention, and read the code, you'll have noticed the UDP stuff I snuck in there. That's because we're going to make this super easy and use UDP to send our data. And there's a <strong>really</strong> good reason for that: InfluxDB Line Protocol. You see, QuestDB has a built-in InfluxDB Line Protocol listener, but (for now) it's only listening on a UDP port. So we're going to use that.
Now, to send some data:
Yeah, there's a lot going on in there. So let's break it down. First, I'm creating a buffer to hold the data I'm going to send, and then I'll do a read of the sensor. I set the `temp` and `hum` variables to values that I know the sensor will never return so that I can check that I got valid readings later, to avoid sending gibberish to the database.
I have to do some shenanigans with the temperature and humidity values in there because one of the shortcomings of Arduinos is that they don't have `sprintf` support for doubles. I know. So I simply turn them into strings and move on. Once they arrive at the database, they are interpreted as doubles and life is good. Not worth fighting about. I can then construct a buffer with straight line protocol and ship it off to QuestDB over UDP.
Don't forget to free the memory!
Remember I told you to set a pointer earlier about creating the database? Well, here's where I come back to that. You don't <em>actually</em> have to create the database ahead of time <em>unless</em> you want to do things like set indexes, etc. If all you want to do is have straight values in there, then guess what? Schema-on-write is a thing here. You can just start writing data to the database, and it will happily store them for you. Pretty cool stuff.</p>
Querying the Data
Using the QuestDB Console, you can then query the data to make sure you're getting what you expect:
That's exactly what I expected!
Now it's time to start building some dashboards, etc. on top of this. I'm currently working on connecting this all up with Node Red, so that may be my next post. We're also working on support for Grafana, which will be huge, so stay tuned for that. If you like what you see here, please go give us a star on GitHub, and follow the project if you'd like to get updates!
Published at DZone with permission of David G. Simmons , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.