Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Redirecting flow based on Certain Condition - Nifi

Solved Go to solution
Highlighted

Redirecting flow based on Certain Condition - Nifi

Contributor

Hi

I have a requirement where I need to redirect a flow based on property value set in parent processor. Below image shows an example of it.

7329-image-076.png

If property is set to ABC then flow file should go to ABC processor. I was looking at RouteOnAttribute but not sure how can i use it.

Thanks

Shashi

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

@Shashi Vish

RouteOnAttribute should be the correct processor to use here. How many possible values are you wanting to route on? If the number is small, you can define each of them as properties within the RouteOnAttribute processor. When you configure the processor, set the routing strategy to Route to Property name. Then add a property for each value you need to deal with, using expression language to evaluate whether it is true.

Here is an example where I'm checking to see if a the twitter.text has 'elasticsearch' or 'solr':

7343-screen-shot-2016-09-03-at-124618-pm.png

Then when I connect the RouteOnAttribute processor to each of my other processors, I can specify the condition on which they should be routed. Here is an example when I connect it to my PutElasticsearch processor:

7344-screen-shot-2016-09-03-at-124329-pm.png

When I created the connection, I only checked the "elasticsearch" relationship. If I create a connection to the PutSolr processor, I might check the 'solr" relationship. In your case, the default processor could be routed on the "unmatched" relationship.

I hope this helps.

View solution in original post

8 REPLIES 8
Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

@Shashi Vish

RouteOnAttribute should be the correct processor to use here. How many possible values are you wanting to route on? If the number is small, you can define each of them as properties within the RouteOnAttribute processor. When you configure the processor, set the routing strategy to Route to Property name. Then add a property for each value you need to deal with, using expression language to evaluate whether it is true.

Here is an example where I'm checking to see if a the twitter.text has 'elasticsearch' or 'solr':

7343-screen-shot-2016-09-03-at-124618-pm.png

Then when I connect the RouteOnAttribute processor to each of my other processors, I can specify the condition on which they should be routed. Here is an example when I connect it to my PutElasticsearch processor:

7344-screen-shot-2016-09-03-at-124329-pm.png

When I created the connection, I only checked the "elasticsearch" relationship. If I create a connection to the PutSolr processor, I might check the 'solr" relationship. In your case, the default processor could be routed on the "unmatched" relationship.

I hope this helps.

View solution in original post

Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

Contributor

@Michael Young thanks for your reply. I have 3 criteria for routing. Just one more i would like to understand. You mentioned here i need to define expression as routing criteria ($twitter.text.contains("elasticsearch")). Can you please tell me from where $twitter.text value will come ? Will that be property of my parent processor?

Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

@Shashi Vish

In my case, the value of twitter.text is set by a processor before the RouteOnAttribute. I use EvaluateJsonPath to pull data out of the flowfile and set properties. You can use expression language to access flow content or flow attributes. Here is a link to the documentation for expression language: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html

In your case, where are these properties? Are they part of the flow file content?

Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

Explorer

@myoung ...in nifi i have used invoked http processor that get data from web app.. i want to apply the condition on json data file ...i means when the resource="abc" the alarm get filter and goes to one processor and other alarm will go to another processor.....so my question is what are the processor is required..as of now i have used one http invoked that will provide all the alarm form my web app ..two putfile processor for storing alarm one for resource="abc" and one for other resource. and i have also used one Routonatrribute processor for condition..in RoutOnAttribute config i have decalred on dynamic property whic is ....the name of property is alarms.resource and the value is ${alarms.resource:equals('abc')} 

renuu_0-1590061592158.png

the below is respon of Invoked http processor...want the alarm that contain resource='abc' that will go to one put file processor
Spoiler
{ "alamrs": [ { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }

Re: Redirecting flow based on Certain Condition - Nifi

Explorer

@myoung ...in nifi i have used invoked http processor that get data from web app.. i want to apply the condition on json data file ...i means when the resource="abc" the alarm get filter and goes to one processor and other alarm will go to another processor.....so my question is what are the processor is required..as of now i have used one http invoked that will provide all the alarm form my web app ..two putfile processor for storing alarm one for resource="abc" and one for other resource. and i have also used one Routonatrribute processor for condition..in RoutOnAttribute config i have decalred on dynamic property whic is ....the name of property is alarms.resource and the value is ${alarms.resource:equals('abc')} 

renuu_0-1590061592158.png

the below is respon of Invoked http processor...want the alarm that contain resource='abc' that will go to one put file processor
Spoiler
{ "alamrs": [ { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }, { "correlate": [ " ", " ", " " ], "event": " ", "group": " ", "history": [ { "event": " ", "href": " ", "id": " ", "resource": " abc ", "status": " ", "text": " ", "type": " ", "updateTime": " ", "user": " ", "value": "--" } ], "href": " ", "id": "", "rawData": null, "receiveTime": " ", "service": [ " " ], "resource": "abc", "status": " ", "tags": [ "" ], "text": " : ", "timeout": , }

Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

Contributor

@Michael Young It worked.

Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

I'm very glad to hear that!

Highlighted

Re: Redirecting flow based on Certain Condition - Nifi

Hi..

I am using a flow which is like GetFile--->SplitText---->ExtractText---->RouteOnAttribute----->PutFile.

In this I have defined certain conditions in extract text as mentioned in the file extracttext.png. It is working fine but i want that after filtering the text from extracttext processor i want to make separate directories depends on the condition like i want for condition1 i want to put it in Folder1, for condition2 i want it on Folder2 and Folder3 for unmatched files.For that i used RouteOnAttribute with following properties as shown in file routeonattribute.png. But it is not working as i am not getting any file in the folders.Can you please tell me what am i doing wrong in it. Here is my flow after routeOnAttribute routingflowbyrouteonattribute.png.

Don't have an account?
Coming from Hortonworks? Activate your account here