Mastering Fluent Bit: Installing Fluent Bit From Source (Part 1)
This intro to mastering Fluent Bit covers what Fluent Bit is, why source installs matter, and how to get it running locally in minutes.
CORE
·
Join the DZone community and get the full member experience.
Join For FreeThis series is a general-purpose getting-started guide for those who want to learn about the Cloud Native Computing Foundation (CNCF) project Fluent Bit.
Each article in this series addresses a single topic by providing insights into what the topic is, why we are interested in exploring that topic, where to get started with the topic, and how to get hands on with learning about the topic as it relates to the Fluent Bit project.
The idea is that each article can stand on its own, but they also lead down a path that slowly increases our abilities to implement solutions with Fluent Bit telemetry pipelines.
Let's take a look a the topic of this article, installing Fluent Bit from the project source code.
This first article kicks off the series, and there is no better place to start than a short introduction and then installing our first ever Fluent Bit telemetry pipeline on our own local machine.
What Is Fluent Bit?
Before diving into Fluent Bit, let's step back and look at the position of this project within the Fluent organization. If we look at the Fluent organization on GitHub, we find Fluentd and Fluent Bit project hosted there. The back story is that the project started with log parsing project Fluentd joining the CNCF in 2026 and reaching Graduated status in 2019.
Once it became apparent that the world was heading into cloud native Kubernetes environments, the solution was not designed for the flexible and lightweight requirements that Kubernetes solutions demanded. Fluent bit was born from the need to have a low-resource using, high-throughput, and highly scalable log management solution for cloud native Kubernetes environments. The project was started within the Fluent organization as a sub-project in 2017, and the rest is now 10 years of history with the release of v4 last week.
Let's get started with Fluent Bit and see what we can do for ourselves.
Why Install From Source Code?
The maintainers of any open-source project will provide the source code project for anyone to access and build from scratch. This is often intimidating and not always straightforward, so if possible I tend to still provide this option to users of my hands-on content. There is nothing easier than testing your Fluent Bit configurations in a development environment or demoing from your local machine with a source built version of the software.
In later articles, I'll provide the ways you can use Fluent Bit container images and build your own versions of those container images.
Where to Get Started
There are always requirements that need to be satisfied on your local machine before you can build any software project, and Fluent Bit is no different. You will need to have the software cmake, openssl, and bison installed and updated to the correct versions.
Due to all of this being time consuming to verify, I've put together a single project you use that runs these checks on your machine and will only proceed if they are all satisfied. Should any checks fail, you are pointed in the direction of a solution.
First, let's use my Fluent Bit Easy Install project to build Fluent Bit from the source code, then I'll follow that with details on each step should you want to verify and build the project from source one step at a time.
How to Install and Configure
The Fluent Bit Easy Install project can be set up and run in just three easy steps:
- Download and unzip the project found here.
- Run the installation script.
- Verify Fluent Bit installation.
After clicking on the download link above, you unzip the downloaded file and move into the project directory:
$ unzip fluentbit-install-demo-v1.8.zip Archive: fluentbit-install-demo-v1.8.zip creating: fluentbit-install-demo-v1.8/ extracting: fluentbit-install-demo-v1.8/.gitignore inflating: fluentbit-install-demo-v1.8/README.md creating: fluentbit-install-demo-v1.8/docs/ creating: fluentbit-install-demo-v1.8/docs/demo-images/ inflating: fluentbit-install-demo-v1.8/docs/demo-images/workshop.png inflating: fluentbit-install-demo-v1.8/init.bat inflating: fluentbit-install-demo-v1.8/init.sh creating: fluentbit-install-demo-v1.8/installs/ inflating: fluentbit-install-demo-v1.8/installs/README inflating: fluentbit-install-demo-v1.8/installs/fluent-bit-4.0.0.zip creating: fluentbit-install-demo-v1.8/support/ ... $ cd fluentbit-install-demo-v1.8
The init.sh file is the installation script you can run with one argument, source, which ensures it's going to build the Fluent Bit project from source and install it for you within the project directory ./target. All of my Easy Install projects promise to not change any default settings on your local machine and to keep all changes within the project directories.
Let's run the installation:
$ ./init.sh source [INFO] ####################################################################### [INFO] ## ## [INFO] ## Setting up the Fluent Bit Easy Install demo ## [INFO] ## ## [INFO] ## ##### # # # ##### # # ##### #### ##### ##### ## [INFO] ## # # # # # ## # # # # # # ## [INFO] ## #### # # # ### # # # # #### # # ## [INFO] ## # # # # # # ## # # # # # ## [INFO] ## # ##### ##### ##### # # # #### ##### # ## [INFO] ## ## [INFO] ## ##### ### #### # # ## [INFO] ## # # # # # # ## [INFO] ## ### ##### ### # ## [INFO] ## # # # # # ## [INFO] ## ##### # # #### # ## [INFO] ## ## [INFO] ## ##### # # #### ##### ### # # ## [INFO] ## # ## # # # # # # # ## [INFO] ## # # # # ### # ##### # # ## [INFO] ## # # ## # # # # # # ## [INFO] ## ##### # # #### # # # ##### ##### ## [INFO] ## ## [INFO] ## brought to you by Eric D. Schabell ## [INFO] ## ## [INFO] ## git@gitlab.com:o11y-workshops/fluentbit-install-demo.git ## [INFO] ## ## [INFO] ####################################################################### [INFO] Checking the build mode arguments... [INFO] Installing from source... [INFO] Installing Fluent Bit version 4.0.0 from the source project... [INFO] Checking for bison version needed to build: [INFO] Bison major version is good... [INFO] Unpacking Fluent Bit project into target directory... [INFO] Fluent Bit project unpacked successfully into target directory! ...
You see the installation start, verification that we are building from source, the version being used, some validation that the right software is available to build, and the source code being unpacked into the project ./target directory.
Next, the first build command is run for phase 1:
... [INFO] Building Fluent Bit (phase 1): -- The C compiler identification is AppleClang 17.0.0.17000013 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done ...
Once that completes, phase 2 of the build is started:
... [INFO] Phase 1 build successful, building continuing with phase 2: [ 0%] Building C object tools/xxd-c/CMakeFiles/xxd-c.dir/xxd-c.c.o [ 0%] Building C object lib/jsmn/CMakeFiles/jsmn.dir/jsmn.c.o [ 0%] Creating directories for 'backtrace' [ 0%] Building C object lib/msgpack-c/CMakeFiles/msgpack-c-static.dir/src/objectc.c.o ... [INFO] Fluent Bit built successfully! [INFO] Copying to ./target/fluent-bit-4.0.0/bin for use. ...
Then we finish up with the build completing successfully and see the details provided to verify that Fluent Bit is installed:
... [INFO] ========================================================= [INFO] = = [INFO] = Install complete, get ready to rock Fluent Bit! = [INFO] = = [INFO] = The binary build is available to run at: = [INFO] = = [INFO] = $ cd target/fluent-bit-4.0.0/bin/fluent-bit = [INFO] = $ bin/fluent-bit --version = [INFO] = = [INFO] = Fluent Bit v4.0.0 = [INFO] = = [INFO] = To get started building your first observability = [INFO] = observability pipelines with real hands on = [INFO] = examples, see below. = [INFO] = = [INFO] = Getting started workshop available online: = [INFO] = https://o11y-workshops.gitlab.io/workshop-fluentbit = [INFO] = = [INFO] =========================================================
The local executable binary can be started with the flag --version, as shown in the console output for now to just verify the build works for you.
Behind the Scenes
While it's nice to see the source code build of a project taken out of your hands, if you're like me, you kind of enjoy knowing what is happening and how to build it by hand. To that end let's walk through the process of building Fluent Bit as done above, but one command at a time on OSX.
First, download and unzip the release. At the time of this writing, that is Fluent Bit v4.0.0:
$ unzip fluent-bit-v4.0.0.zip [EXPANDING THE ZIP FILE HERE] $ cd fluent-bit-v4.0.0/build $ cmake ../ -DOPENSSL_ROOT_DIR=$(brew --prefix openssl) -DCMAKE_INSTALL_PREFIX=../bin/fluent-bit ... [CUT-LOTS-OF-BUILD-LINES] -- include: lib/simdutf-amalgamation-5.5.0/src/simdutf -- Configuring done (34.7s) -- Generating done (0.9s) -- Build files have been written to: [PATH-TO-PROJECT]/fluent-bit-4.0.0/build
That was the completion of phase 1 of the Fluent Bit build. Now to kick off phase 2 with the next command:
$ make -j 16 ... [CUT-LOTS-OF-BUILD-LINES] [100%] Built target out_lib [100%] Built target hello_world
This is the completion of phase 2 of the Fluent Bit build. Now to install it you run the following command:
$ make install ... [CUT-LOTS-OF-INSTALL-LINES] -- Installing: [PATH-TO-PROJECT]/fluent-bit-4.0.0/bin/fluent-bit/bin/fluent-bit -- Installing: [PATH-TO-PROJECT]/fluent-bit-4.0.0/bin/fluent-bit/etc/fluent-bit/fluent-bit.conf -- Installing: [PATH-TO-PROJECT]/fluent-bit-4.0.0/bin/fluent-bit/etc/fluent-bit/parsers.conf -- Installing: [PATH-TO-PROJECT]/fluent-bit-4.0.0/bin/fluent-bit/etc/fluent-bit/plugins.conf
This completes the installation of your source build, still installing only within the confines of the Fluent Bit source directories. You will find the Fluent Bit executable in the project directory and can verify using the following command:
# From the fluent-bit-4.0.0 root directory # $ ./bin/fluent-bit/bin/fluent-bit --version Fluent Bit v4.0.0
There you have it, your very own source build of Fluent Bit.
More in the Series
In this article, you learned how to build Fluent Bit from source, verify that it worked, and were walked through doing it step-by-step. This article is based on this online free workshop with a lab to install Fluent Bit from source.
There will be more in this series as you continue to learn how to configure, run, manage, and master the use of Fluent Bit in the wild. Next up, installing Fluent Bit using container images.
Published at DZone with permission of Eric D. Schabell, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments