Building a Simple Kafka Client for the Web and Desktop
This article is about building a simple Kafka real-time web client using KafkaJS, Socket.io, and ElectronJS.
Join the DZone community and get the full member experience.Join For Free
Kafka is an open-source platform that is used for event distribution, logging, and streaming. Distributions from Confluent and other vendors provide a web UI for broker management and client tasks, while those from Apache Kafka provide just command-line tools. Also, for most of the requirements, a simple client that publishes and consumes events or messages from the topic is sufficient. The main objectives of building such a client were:
- Follow a web development approach
- Have a simple and elegant UI
- Build a native desktop app
Being a sort of a de facto standard for most web development with a huge selection of frameworks and libraries, node.js was chosen as the platform. The UI framework was Bootstrap with a Material theme. Though some might argue that it is dated and has lesser capabilities, it fit the bill for the requirement in question. ElectronJS was used to provide the native desktop app capabilities. It has a flat learning curve with usually only a main file that it generates with functionality to manage the window being displayed. The Kafka client-specific code can be in separate files and can be included in this file. Using utilities such as electron packager, the executable code for the respective platform can be generated.
- Connecting automatically to a Kafka broker
- Creating a topic on the fly
- Publishing and consuming messages
To provide a real-time view of the messages in the Kafka topic, Socket.io is used. It enables real-time communication using WebSockets from the Kafka topic to the UI. The consumer sees the messages on the topic it has subscribed to in the UI in real-time as they get published. The code to implement comprises adding a line of code to emit the event at the server and to consume it in the UI and display it.
Adding new features to the Kafka client or making it available on more platforms is not as difficult as the underlying libraries and frameworks are robust and well adopted. This approach for building a real-time web client be used for messaging platforms such as RabbitMQ, Solace, and others.
Opinions expressed by DZone contributors are their own.