Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Apache NiFi With Rule Engine

DZone's Guide to

Apache NiFi With Rule Engine

In this quick post, we take a look at how to use Apache NiFi, and open source big data platform, to perform data manipulation.

· Big Data Zone ·
Free Resource

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

Combining Apache NiFi with Rule engine is easy and fun. You can do some really great stuff with data manipulation, and I wrote some code for that purpose using Drools.

Build or Download Rule Engine

Download a compiled NiFi NAR here or Build from https://github.com/alefbt/NiFi-Rule-engine-processor

This is currently working only with JSON flow files.

(Licence: Apache2)

NiFi-Rule-engine-processor github

Then run the following: 

git clone https://github.com/alefbt/NiFi-Rule-engine-processor
cd NiFi-Rule-engine-processor
mvn clean install package
cp nifi-ruleengien-processor-nar/target/nifi-ruleengien-processor-1.0.nar /path-to-nifi/lib/

And (re)start NiFi.

Example

Create a NiFi Flow

  1. GetFile Processor
    • Set property `source folder` : /some-project-path/IN
  2. RuleEngineProcessor
    • Set the property DRL file path to /some-project-path/DRL/business_object_json_test1.drl
  3. PutFile Processor
    • Set the property `dest folder` to /some-project-path/OUT

Image title

Create Files and Relevant Folders

Image title

Create a DRL file /some-project-path/DRL/business_object_json_test1.drl

package com.matrixbi.rules

import com.matrixbi.objects.BusinessObject

rule "Good Morning"
when
b: BusinessObject()
eval( b.getAsInt("time") < 12 )
then
b.set("greet","G00d Morning " + b.get("name")); 
end

rule "Good Afternoon"
when
b: BusinessObject()
eval(b.getAsInt("time") >=12 && b.getAsInt("time") < 16)
then
b.set("greet","Good Afternoon " + b.get("name"));
end

rule "Good Night"
when
b: BusinessObject()
eval(b.getAsInt("time") >= 16)
then
b.set("greet","Good Afternoon " + b.get("name"));
end

Create a JSON test file /some-project-path/IN/business_object_json_test1.json

{
"time": 11,
"greet": "Default text",
"name":"Yehuda"
}

Activate all and run it.

Result

After a few moments, you should see business_object_json_test1.json in the/some-project-path/OUT folder file.

Image title

Source

Image titleResult

Image title

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.

Topics:
apache nifi ,drools ,rule engine ,big data ,tutorial

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}