Mastering Fluent Bit: Developer Guide to Telemetry Pipeline Routing (Part 12)

This series is a general-purpose getting-started guide for those who want to learn about the Cloud Native Computing Foundation (CNCF) project Fluent Bit.

Each article in this series addresses a single topic by providing insights into what the topic is, why it is worth exploring, where to get started, and how to get hands-on with learning about the topic as it relates to the Fluent Bit project.

The idea is that each article can stand on its own, but that they also lead down a path that slowly increases our abilities to implement solutions with Fluent Bit telemetry pipelines.

In this article, we focus on using Fluent Bit routing for developers. If you missed the previous article, check out The Top Three Telemetry Pipeline Filters, where you explored the most useful filters for manipulating and controlling telemetry data.

This article is a hands-on exploration of routing patterns that help developers build sophisticated telemetry pipelines. We will examine how to direct telemetry data to different destinations based on tags, patterns, and conditions in Fluent Bit configurations.

All examples in this article were created on macOS (OSX) and assume the reader can adapt the demonstrated actions to their own local environments.

You should have completed the previous articles in this series to install and get started with Fluent Bit on your local development machine, either using the source code or container images. Links at the end of this article point to a free, hands-on workshop that explores Fluent Bit in more detail.

You can verify that you have a functioning Fluent Bit installation by running the following tests, either from a source installation or a container-based installation:

# For source installation.
$ fluent-bit -i dummy -o stdout

# For container installation.
$ podman run -ti ghcr.io/fluent/fluent-bit:4.2.0 -i dummy -o stdout

...
[0] dummy.0: [[1753105021.031338000, {}], {"message"=>"dummy"}]
[0] dummy.0: [[1753105022.033205000, {}], {"message"=>"dummy"}]
[0] dummy.0: [[1753105023.032600000, {}], {"message"=>"dummy"}]
[0] dummy.0: [[1753105024.033517000, {}], {"message"=>"dummy"}]
...

Now, let’s explore how routing works in Fluent Bit and why it is essential for building production-ready telemetry pipelines.

Understanding Routing in Telemetry Pipelines

Refer to the linked article for details about the service section of the configuration used throughout this article. For now, we focus on the Fluent Bit pipeline itself — specifically the routing capabilities that allow us to direct telemetry data to appropriate destinations.

The telemetry pipeline consists of several phases. Routing occurs throughout the pipeline but is most visible in the final output phase, where we decide which processed events are sent to which destinations.

Routing in Fluent Bit determines which events flow to which outputs. There are two primary routing approaches available to developers:

Tag-Based Routing

Every event in Fluent Bit carries a tag, and outputs use the match parameter to subscribe to events with specific tags. Tags are simple strings that act as identifiers for event streams, similar to topics in message queues. This tag-based routing system is the traditional foundation of Fluent Bit’s routing capabilities.

Tags typically follow a hierarchical naming convention, such as app.frontend.logs or system.metrics.cpu. This structure makes it easy to create wildcard patterns for flexible matching.

The match parameter supports wildcards:

• A single asterisk (*) matches any characters at one level.
• A double asterisk (**) matches multiple levels of the hierarchy.

For example:

• app.* matches app.logs and app.metrics, but not app.frontend.logs.
• app.** matches all of them.

This wildcard system provides the flexibility needed for complex routing scenarios while keeping configurations manageable.

Conditional Routing

Conditional routing is a newer feature available in Fluent Bit 4.2 and later. It evaluates individual records and routes them to different outputs based on their content. Unlike tag-based routing, which operates on entire chunks of data, conditional routing enables per-record routing decisions.

This mechanism uses a routes block defined within input configurations. Each route specifies conditions that determine which records match and which outputs matching records should be sent to.

Conditions can evaluate any field within a record using operators such as equality, greater-than comparisons, regular expressions, and array membership tests.

Conditional routing is particularly powerful when you need to:

• Split logs based on severity
• Route records from different services to dedicated outputs
• Implement complex multi-condition routing logic

It provides fine-grained control without requiring filters to modify tags, making routing configurations more explicit and easier to understand.

The key advantage is that routing decisions are made per-record at the input stage, before any filtering or processing occurs. This enables efficient routing that can send subsets of data to different destinations based on content, without processing or forwarding unnecessary records.

Combining Routing Mechanisms

In production environments, you rarely send all your telemetry data to a single destination. Different types of logs need different handling. Error logs might go to an alerting system, audit logs to long-term storage, and debug logs might be dropped entirely to save costs. 

You can combine both routing approaches in a single configuration. Use tag-based routing for broad categorization by source or input type, and conditional routing for fine-grained decisions based on record content like severity level, error codes, or specific field values. Together, these routing approaches give you complete control over your telemetry pipeline.

Understanding these routing patterns is essential for developers who want to build sophisticated telemetry pipelines that efficiently direct data where it needs to go while minimizing costs and maximizing value.

Now let's look at practical implementations of these routing patterns that developers need to master.

Basic Tag-Based Routing with Wildcards

The traditional routing mechanism in Fluent Bit is the tag-based system. Every input assigns data a human-readable identifier (a tag), and every output uses a match pattern to select which events it receives.

Let’s create a configuration that demonstrates basic routing patterns. First, create a configuration file called fluent-bit.yaml as follows:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: app.frontend.requests
      dummy: '{"service":"frontend","level":"info","message":"Request processed"}'

    - name: dummy
      tag: app.backend.errors
      dummy: '{"service":"backend","level":"error","message":"Database connection failed"}'
    
    - name: dummy
      tag: system.metrics
      dummy: '{"cpu_usage":75,"memory_usage":60}'

  outputs:
    - name: stdout
      match: app.frontend.*
      format: json_lines
    
    - name: stdout
      match: app.backend.*
      format: json_lines
    
    - name: stdout
      match: system.*
      format: json_lines

This configuration creates three different dummy inputs, each with its own tag representing different sources of telemetry data. The tag structure uses a hierarchical naming convention — similar to Java package names or DNS domains — which makes it easy to create wildcard patterns.

The outputs section defines three stdout outputs, each matching a different tag pattern. The asterisk (*) wildcard matches any characters at that level. This means app.frontend.* matches any event with a tag starting with app.frontend., regardless of what follows.

Let’s run this configuration to see tag-based routing in action:

# For source installation.

$ fluent-bit --config fluent-bit.yaml



# For container installation after building new image with your 

# configuration using a Buildfile as follows:

#

# FROM ghcr.io/fluent/fluent-bit:4.2.0

# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml

# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]

#

$ podman build -t fb -f Buildfile



$ podman run --rm fb 



...

{"date":"2025-12-18 10:15:23.456789","service":"frontend","level":"info","message":"Request processed"}
{"date":"2025-12-18 10:15:24.567890","service":"backend","level":"error","message":"Database connection failed"}
{"date":"2025-12-18 10:15:25.678901","cpu_usage":75,"memory_usage":60}

...

Each event is routed to its corresponding output based on the tag match pattern. This basic routing pattern forms the foundation for more complex scenarios. In these examples, we use stdout for simplicity, leaving it to the reader to experiment with other output destinations.

You can also use the double asterisk (**) wildcard to match multiple tag levels. Let’s modify the outputs to demonstrate this:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: app.frontend.requests
      dummy: '{"service":"frontend","level":"info","message":"Request processed"}'

    - name: dummy
      tag: app.backend.errors
      dummy: '{"service":"backend","level":"error","message":"Database connection failed"}'
    
    - name: dummy
      tag: system.metrics
      dummy: '{"cpu_usage":75,"memory_usage":60}'

  outputs:
    - name: stdout
      match: app.**
      format: json_lines

Running this configuration shows that the single output with app.** matches both app.frontend.requests and app.backend.errors, but not system.metrics. The double asterisk matches any number of tag levels, making it ideal for catching all events under a hierarchical namespace.

Conditional Routing

While tag-based routing directs entire chunks of data based on their source, conditional routing evaluates individual records and makes routing decisions based on their content. This feature, available in Fluent Bit 4.2 and later, uses a routes block within input configurations to define routing rules.

Let’s create a configuration that routes logs to different destinations based on severity level:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: app.logs
      dummy: '{"service":"api","level":"ERROR","message":"Failed to connect"}'
      routes:
        logs:
          - name: critical_errors
            condition:
              op: or
              rules:
                - field: "$level"
                  op: eq
                  value: "ERROR"
                - field: "$level"
                  op: eq
                  value: "FATAL"
            to:
              outputs:
                - critical_output
    
    - name: dummy
      tag: app.logs
      dummy: '{"service":"api","level":"INFO","message":"Request completed"}'
      routes:
        logs:
          - name: normal_logs
            condition:
              op: and
              rules:
                - field: "$level"
                  op: eq
                  value: "INFO"
            to:
              outputs:
                - info_output

  outputs:
    - name: stdout
      alias: critical_output
      format: json_lines
    
    - name: stdout
      alias: info_output
      format: json_lines

The routes block is where conditional routing configuration happens. Each route has several key components:

• name – a unique identifier for the route
• condition – the logic block that determines which records match
• condition.op – the logical operator (and / or) for combining multiple rules
• condition.rules – an array of rules to evaluate against each record
• to.outputs – destination outputs (by name or alias) for matching records

Each rule in the condition.rules array specifies:

• field – the field to examine using record accessor syntax ($level, $service, etc.)
• op – the comparison operator (eq, neq, gt, lt, gte, lte, regex, not_regex, in, not_in)
• value – the value to compare against (a single value or an array for in/not_in)
• context (optional) – where to look for the field (body, metadata, otel_resource_attributes, etc.)

When a record arrives, Fluent Bit evaluates the conditions for each route in order. Records are sent to the outputs whose conditions they match. This enables per-record routing decisions based on the actual content of each log entry.

Let’s run this configuration:

# For source installation.

$ fluent-bit --config fluent-bit.yaml



# For container installation after building new image with your 

# configuration using a Buildfile as follows:

#

# FROM ghcr.io/fluent/fluent-bit:4.2.0

# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml

# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]

#

$ podman build -t fb -f Buildfile



$ podman run --rm fb 



...

{"date":"2025-12-18 10:30:45.123456","service":"api","level":"ERROR","message":"Failed to connect"}
{"date":"2025-12-18 10:30:46.234567","service":"api","level":"INFO","message":"Request completed"}

...

The ERROR logs are routed to critical_output, while INFO logs are routed to info_output, based on the value of the level field in each record.

Default Routes

You can also define a default route to catch records that do not match any other conditions:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: app.logs
      dummy: '{"service":"api","level":"ERROR","message":"Failed to connect"}'
      routes:
        logs:
          - name: error_logs
            condition:
              op: and
              rules:
                - field: "$level"
                  op: eq
                  value: "ERROR"
            to:
              outputs:
                - error_output
          
          - name: default_logs
            condition:
              default: true
            to:
              outputs:
                - default_output

  outputs:
    - name: stdout
      alias: error_output
      format: json_lines
    
    - name: stdout
      alias: default_output
      format: json_lines

The route with condition.default: true acts as a catch-all for any records that do not match earlier routes.

Multi-Condition Routing

Here is a more complex example that routes records based on multiple conditions:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: app.logs
      dummy: '{"service":"api","level":"ERROR","environment":"production","response_time":6000}'
      routes:
        logs:
          - name: high_priority_errors
            condition:
              op: and
              rules:
                - field: "$level"
                  op: eq
                  value: "ERROR"
                - field: "$environment"
                  op: eq
                  value: "production"
                - field: "$response_time"
                  op: gt
                  value: 5000
            to:
              outputs:
                - critical_output
                - audit_output

          - name: all_other_logs
            condition:
              default: true
            to:
              outputs:
                - general_output

  outputs:
    - name: stdout
      alias: critical_output
      format: json_lines
    
    - name: stdout
      alias: audit_output
      format: json_lines

    - name: stdout
      alias: general_output
      format: json_lines

This configuration uses the and operator to combine three conditions. Only records that are ERROR level, from production, AND have a response time greater than 5000ms are routed to both the critical_output and audit_output. All other records go to general_output.

Let's run this configuration:

# For source installation.

$ fluent-bit --config fluent-bit.yaml



# For container installation after building new image with your 

# configuration using a Buildfile as follows:

#

# FROM ghcr.io/fluent/fluent-bit:4.2.0

# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml

# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]

#

$ podman build -t fb -f Buildfile

$ podman run --rm fb \



...

{"date":"2025-12-18 10:45:12.345678","service":"api","level":"ERROR","environment":"production","response_time":6000}
{"date":"2025-12-18 10:45:12.345678","service":"api","level":"ERROR","environment":"production","response_time":6000}
...

Notice how the high-priority error appears twice in the output because it's being sent to two different outputs (critical_output and audit_output).

Matching Multiple Values with in

Conditional routing also supports the in and not_in operators for matching against arrays of values:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: app.logs
      dummy: '{"service":"payment","level":"ERROR","message":"Transaction failed"}'
      routes:
        logs:
          - name: critical_services
            condition:
              op: and
              rules:
                - field: "$service"
                  op: in
                  value: ["payment", "authentication", "database"]
            to:
              outputs:
                - critical_output

          - name: standard_logs
            condition:
              default: true
            to:
              outputs:
                - standard_output

  outputs:
    - name: stdout
      alias: critical_output
      format: json_lines
    
    - name: stdout
      alias: standard_output
      format: json_lines

This configuration routes logs from critical services (payment, authentication, database) to a dedicated output, while all other services go to standard output. The in operator makes it easy to match against multiple values without creating separate rules for each one.

Advanced Routing with Regular Expressions and Nested Tags

Advanced routing can also be implemented using tag-based fan-out, where a single event is sent to multiple destinations by having multiple outputs with overlapping match patterns:

service:
  flush: 1
  log_level: info
  http_server: on
  http_listen: 0.0.0.0
  http_port: 2020
  hot_reload: on

pipeline:
  inputs:
    - name: dummy
      tag: prod.svc.auth.logs
      dummy: '{"service":"auth","environment":"production","level":"ERROR","message":"Token validation failed"}'

  outputs:
    - name: stdout
      match: 'prod.**'
      format: json_lines
      
    - name: stdout
      match_regex: '.*\.auth\..*'
      format: json_lines
    
    - name: stdout
      match: '**.logs'
      format: json_lines

Running this configuration shows that the single input event matches all three output patterns:

# For source installation.

$ fluent-bit --config fluent-bit.yaml

# For container installation after building new image with your

# configuration using a Buildfile as follows:

#

# FROM ghcr.io/fluent/fluent-bit:4.2.0

# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml

# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]

#

$ podman build -t fb -f Buildfile

$ podman run --rm fb 



...

{"date":"2025-12-18 11:15:45.567890","service":"auth","environment":"production","level":"ERROR","message":"Token validation failed"}
{"date":"2025-12-18 11:15:45.567890","service":"auth","environment":"production","level":"ERROR","message":"Token validation failed"}
{"date":"2025-12-18 11:15:45.567890","service":"auth","environment":"production","level":"ERROR","message":"Token validation failed"}
...

The event appears three times because it matches:

• prod.** (production environment)
• .*\.auth\..* (auth service)
• **.logs (logs data type)

This fan-out pattern is essential for implementing multi-destination routing without duplicating configuration or using additional filters.

Important Routing Considerations

When implementing routing in Fluent Bit pipelines, keep these points in mind:

• Design tag structures carefully: Tags are the backbone of your routing strategy. Use hierarchical naming with clear semantic levels. This makes it easy to create flexible match patterns.
• Be aware of ordering: Outputs are processed in the order they are defined. This usually does not matter but can affect advanced scenarios.
• Use match patterns efficiently: Prefer wildcard patterns over many exact matches to keep configurations maintainable.
• Consider performance implications: Every output adds overhead. Test performance impact and consolidate outputs when possible.
• Test routing logic: Use stdout during development to verify routing behavior and catch errors early.
• Document routing patterns: Complex routing configurations can be difficult to understand months later. Add comments explaining your tag structure and routing logic to help future maintainers.

This covers the essential routing patterns that developers need to master when building Fluent Bit telemetry pipelines for their applications.

More in the Series

In this article, you learned how to implement both simple and advanced routing patterns in Fluent Bit using tags, wildcards, and content-based routing. This content is based on a free online workshop that includes a lab focused on advanced telemetry routing.

More articles are coming as you continue learning how to configure, run, manage, and master Fluent Bit in real-world environments. Next up: exploring Fluent Bit buffering and reliability patterns to ensure your telemetry data is never lost.