Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Apache NiFi With Rule Engine

DZone 's Guide to

Apache NiFi With Rule Engine

In this quick post, we take a look at how to use Apache NiFi, and open source big data platform, to perform data manipulation.

· Big Data Zone ·
Free Resource

Combining Apache NiFi with Rule engine is easy and fun. You can do some really great stuff with data manipulation, and I wrote some code for that purpose using Drools.

Build or Download Rule Engine

Download a compiled NiFi NAR here or Build from https://github.com/alefbt/NiFi-Rule-engine-processor

This is currently working only with JSON flow files.

(Licence: Apache2)

NiFi-Rule-engine-processor github

Then run the following: 

git clone https://github.com/alefbt/NiFi-Rule-engine-processor
cd NiFi-Rule-engine-processor
mvn clean install package
cp nifi-ruleengien-processor-nar/target/nifi-ruleengien-processor-1.0.nar /path-to-nifi/lib/

And (re)start NiFi.

Example

Create a NiFi Flow

  1. GetFile Processor
    • Set property `source folder` : /some-project-path/IN
  2. RuleEngineProcessor
    • Set the property DRL file path to /some-project-path/DRL/business_object_json_test1.drl
  3. PutFile Processor
    • Set the property `dest folder` to /some-project-path/OUT

Image title

Create Files and Relevant Folders

Image title

Create a DRL file /some-project-path/DRL/business_object_json_test1.drl

package com.matrixbi.rules

import com.matrixbi.objects.BusinessObject

rule "Good Morning"
when
b: BusinessObject()
eval( b.getAsInt("time") < 12 )
then
b.set("greet","G00d Morning " + b.get("name")); 
end

rule "Good Afternoon"
when
b: BusinessObject()
eval(b.getAsInt("time") >=12 && b.getAsInt("time") < 16)
then
b.set("greet","Good Afternoon " + b.get("name"));
end

rule "Good Night"
when
b: BusinessObject()
eval(b.getAsInt("time") >= 16)
then
b.set("greet","Good Afternoon " + b.get("name"));
end

Create a JSON test file /some-project-path/IN/business_object_json_test1.json

{
"time": 11,
"greet": "Default text",
"name":"Yehuda"
}

Activate all and run it.

Result

After a few moments, you should see business_object_json_test1.json in the/some-project-path/OUT folder file.

Image title

Source

Image titleResult

Image title

Topics:
apache nifi ,drools ,rule engine ,big data ,tutorial

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}