Streaming ESP32-CAM Images to Multiple Browsers via MQTT

In this tutorial, you'll learn how to publish images from an ESP32-CAM board to multiple browser clients using MQTT (Message Queuing Telemetry Transport). This setup will enable you to create a platform that functions similarly to a live video stream, viewable by an unlimited number of users.

Prerequisites

Before diving in, make sure you have completed the following prerequisite tutorials:

• Your First Xedge32 Project: This tutorial covers essential setup and configuration instructions for Xedge32 running on an ESP32.
• Your First MQTT Lua Program: This tutorial introduces the basics of MQTT and how to write a simple Lua program to interact with MQTT.

By building on the knowledge gained from these foundational tutorials, you'll be better equipped to follow along with this tutorial.

Publishing ESP32-CAM Images via MQTT

In the MQTT CAM code, our primary focus is publishing images without subscribing to other events. This publishing operation is managed by a timer event, which publishes images based on the intervals specified.

Setting Up the Timer

First, let's create a timer object. This timer will trigger the publishImage function at specific intervals.

timer = ba.timer(publishImage)

To interact with the ESP32 camera, initialize a camera object like so:

cam = esp32.cam(cfg)

The cfg parameter represents a configuration table. Important: make sure it matches the settings for your particular ESP32-CAM module. See the Lua CAM API for details.

Handling MQTT Connection Status

For monitoring MQTT connections, use the following callback function:

local function onstatus(type, code, status)
   if "mqtt" == type and "connect" == code and 0 == status.reasoncode then
      timer:set(300, false, true)  -- Activate timer every 300 milliseconds
      return true  -- Accept connection
   end
   timer:cancel()
   return true  -- Keep trying
end

The above function starts the timer when a successful MQTT connection is made. If the connection drops, it cancels the timer but will keep attempting to reconnect.

Image Publishing via Timer Callback

The core of the image publishing mechanism is the timer callback function, publishImage. This function captures an image using the camera object and publishes it via MQTT. The timer logic supports various timer types. Notably, this version operates as a Lua coroutine (akin to a thread). Within this coroutine, it continually loops and hibernates for the duration defined by the timer through coroutine.yield(true).

function publishImage()
   local busy = false
   while true do
      if mqtt:status() < 2 and not busy then
         busy = true -- thread busy
         ba.thread.run(function()
            local image = cam:read()
            mqtt:publish(topic, image)
            busy = false -- no longer running
         end)
      end
      coroutine.yield(true)  -- sleep
   end
end

The above function maintains flow control by not publishing an image if two images already populate the MQTT client's send queue. The cam:read function can be time-consuming -- not in human time, but in terms of microcontroller operations. As such, we offload the task of reading from the CAM object onto a separate thread. While this step isn't strictly necessary, it enhances the performance of applications juggling multiple operations alongside reading from the CAM. For a deeper dive into the threading intricacies, you are encouraged to refer to the Barracuda App Server’s documentation on threading.

The following shows the complete MQTT CAM code:

local topic = "/xedge32/espcam/USA/92629"
local broker = "broker.hivemq.com"
 
-- Settings for 'FREENOVE ESP32-S3 WROOM' CAM board
local cfg={
   d0=11, d1=9, d2=8, d3=10, d4=12, d5=18, d6=17, d7=16,
   xclk=15, pclk=13, vsync=6, href=7, sda=4, scl=5, pwdn=-1,
   reset=-1, freq="20000000", frame="HD"
}
 
-- Open the cam
local cam,err=esp32.cam(cfg)
assert(cam, err) -- Throws error if 'cfg' incorrect
 
local timer -- Timer object; set below.
 
-- MQTT connect/disconnect callback
local function onstatus(type,code,status)
   -- If connecting to broker succeeded
   if "mqtt" == type and "connect" == code and 0 == status.reasoncode then
      timer:set(300,false,true) -- Activate timer every 300 milliseconds
      trace"Connected"
      return true -- Accept connection
   end
   timer:cancel()
   trace("Disconnect or connect failed",type,code)
   return true -- Keep trying
end
 
-- Create MQTT client
local mqtt=require("mqttc").create(broker,onstatus)
 
-- Timer coroutine function activated every 300 millisecond
function publishImage()
   local busy=false
   while true do
      --trace(mqtt:status(), busy)
      -- Flow control: If less than 2 queued MQTT messages
      if mqtt:status() < 2 and not busy then
         busy=true
         ba.thread.run(function()
            local image,err=cam:read()
            if image then
               mqtt:publish(topic,image)
            else
               trace("cam:read()",err)
            end
            busy=false
         end)
      end
      coroutine.yield(true) -- sleep
   end
end
timer = ba.timer(publishImage)

While we have already covered the majority of the program's functionality, there are a few aspects we haven't touched upon yet:

• Topic and Broker Configuration:

  • local topic = "/xedge32/espcam/USA/92629": Sets the MQTT topic where the images will be published. Change this topic to your address.
  • local broker = "broker.hivemq.com": Specifies the MQTT broker's address. The public HiveMQ broker is used in this example.

• ESP32 Camera Configuration (cfg): This block sets up the specific pin configurations and settings for your ESP32 CAM board. Replace these settings with those appropriate for your hardware.

• Creating the MQTT Client: The MQTT client is created with the require("mqttc").create(broker, onstatus) function, passing in the broker address and the onstatus callback.

• Creating the Timer Object for publishImage: The timer is created by calling ba.timer and passing in the publishImage callback, which will be activated at regular intervals. This is the mechanism that continually captures and publishes images.

Subscribing to CAM Images With a JavaScript-Powered HTML Client

To visualize the images published by the ESP32 camera, you can use an HTML client. The following client will subscribe to the same MQTT topic to which the camera is publishing images. The client runs purely in your web browser and does not require any server setup.

The entire code for the HTML client is shown below:

<!DOCTYPE html>
<html lang="en">
<head>
  <title>Cam Images Over MQTT</title>
  <script data-fr-src="https://cdnjs.cloudflare.com/ajax/libs/mqtt/5.0.0-beta.3/mqtt.min.js"></script>
<script>
 
const topic="/xedge32/espcam/USA/92629";
const broker="broker.hivemq.com";
 
window.addEventListener("load", (event) => {
    let img = document.getElementById("image");
    let msg = document.getElementById("msg");
    let frameCounter=0;
    const options = {
        clean: true,
        connectTimeout: 4000,
        port: 8884 // Secure websocket port
    };
    const client = mqtt.connect("mqtts://"+broker+"/mqtt",options);
 
    client.on('connect', function () {
        msg.textContent="Connected; Waiting for images...";
        client.subscribe(topic);
    });
     
    client.on("message", (topic, message) => {
        const blob = new Blob([message], { type: 'image/jpeg' });
        img.src = URL.createObjectURL(blob);
        frameCounter++;
        msg.textContent = `Frames: ${frameCounter}`;
    });
});
</script>
</head>
<body>
    <h2>Cam Images Over MQTT</h2>
   <div id="image-container">
    <img id="image"/>
  </div>
    <p id="msg">Connecting...</p>
</body>
</html>

MQTT JavaScript Client

At the top of the HTML file, the MQTT JavaScript library is imported to enable MQTT functionalities. This is found within the <script data-fr-src=".......mqtt.min.js"></script> line.

Body Layout

The HTML body contains a <div> element with an id of "image-container" that will house the incoming images, and a <p> element with an id of "msg" that serves as a placeholder for status messages.

MQTT Configuration

In the JavaScript section, two constants topic and broker are defined. These must correspond to the topic and broker configurations in your mqttcam.xlua file.

Connecting to MQTT Broker

The client initiates an MQTT connection to the specified broker using the mqtt.connect() method. It uses a secure websocket port 8884 for this connection.

Handling Incoming Messages

Upon a successful connection, the client subscribes to the topic. Any incoming message on this topic is expected to be a binary JPEG image. The message is converted into a Blob and displayed as the source for the image element.

Frame Counter

A frameCounter variable keeps count of the incoming frames (or images) and displays this count as a text message below the displayed image.

By having this HTML file open in a web browser, you'll be able to visualize in real-time the images that are being published to the specified MQTT topic.

Preparing the Code

Step 1: Prepare the Lua Script as Follows

1. As explained in the tutorial Your First Xedge32 Project, when the Xedge32 powered ESP32 is running, use a browser and navigate to the Xedge IDE.
2. Create a new Xedge app called "cam" and LSP enable the app.
3. Expand the cam app now visible in the left pane tree view.
4. Right-click the cam app and click New File in the context menu.
5. Type camtest.lsp and click Enter.
6. Open the camtest.lsp file at GitHub and click the copy raw file button.
7. Go to the Xedge IDE browser window and paste the content into the camtest.lsp file.
8. Important: Adjust the cfg settings in camtest.lsp to match your specific ESP32 CAM board. See the Lua CAM API for details.
9. Click Save and then Click Open to test your cam settings. Make sure you see the image generated by the LSP script before proceeding.
10. Right-click the cam app and click New File in the context menu.
11. Type mqttcam.xlua and click Enter.
12. Open the mqttcam.xlua file at GitHub and click the copy raw file button.
13. Go to the Xedge IDE browser window and paste the content into the mqttcam.xlua file.
14. Using the Xedge editor, update the topic variable /xedge32/espcam/USA/92629 in the Lua script to your desired MQTT topic.
15. Important: Copy the cfg settings from camtest.lsp and replace the cfg settings in mqttcam.xlua with the settings you tested in step 9.
16. Click the Save & Run button to save and start the example.

Step 2: Prepare the HTML/JS File as follows

1. Download mqttcam.html, open the file in any editor, and ensure the topic in the HTML file matches the topic you set in the Lua script.
2. Save the mqttcam.html file.
3. Open mqttcam.html: Double-click the mqttcam.html file or drag and drop it into your browser. Note: this file is designed to be opened directly from the file system. You do not need a web server to host this file.
4. Observe the Output: The webpage will display the images being published by the ESP32 CAM. The number of frames received will be displayed below the image.

Potential Issues with ESP32 CAM Boards and Solutions

ESP32 CAM boards are widely recognized for their versatility and affordability. However, they're not without their challenges. One of the significant issues users might face with the ESP32 CAM boards is interference between the camera read operation and the built-in WiFi module. Let's delve into the specifics:

Problem: Interference and WiFi Degradation

When the ESP32 CAM board is in operation, especially during the camera's read operation, it can generate noise. This noise interferes with the built-in WiFi, which results in:

• Reduced Range: The distance over which the WiFi can effectively transmit and receive data can be notably decreased.
• Decreased Throughput: The speed and efficiency at which data is transmitted over the WiFi network can be considerably hampered.

Solutions

To combat these issues, consider the following solutions:

1. Use a CAM Board with an External Antenna: Several ESP32 CAM boards come equipped with or support the use of an external antenna. By using such a board and connecting an external antenna, you can boost the WiFi signal strength and range, mitigating some of the interference caused by the camera operations.
2. Integrate the W5500 Ethernet Chip: If your application demands consistent and robust data transmission, consider incorporating the W5500 Ethernet chip. By utilizing Ethernet over WiFi, you are effectively sidestepping the interference issues associated with WiFi on the ESP32 CAM board. Xedge32 is equipped with integrated Ethernet drivers. When paired with hardware that supports it, like the W5500 chip, it can facilitate smooth and interference-free data transfer, ensuring that your application remains stable and efficient.

In conclusion, while the ESP32 CAM board is an excellent tool for a myriad of applications, it's crucial to be aware of its limitations and know how to circumvent them to ensure optimal performance.

References

• Lua MQTT API
• Lua timer API
• Lua CAM API