Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Delete/Stop flow files in execute script

Hi Team,

I want to either delete or stop the flow files which didn't satisfy the condition in execute script.

I'm getting lot of flow files into my execute script processor out of which didn't satisfy the condition needs to be stopped/deleted at the execute script processor it self and whichever the flow files satisfy the condition needs to move to the next step. Can we stop/delete the flow files in execute script processor.

Below is the logic inside the execute script.

64639-screen-shot-2018-03-16-at-54506-pm.png

1 ACCEPTED SOLUTION

@Pavan M You can use the remove method from the session object. Follows an example.

NiFi Flow

64662-screen-shot-2018-03-17-at-24346-am.png

Each GenerateFlowFile processor generate an empty flow with an attributed called "a" valued 1, 2 & 3 respectively.

Script in ExecuteScript processor

flowFile = session.get() 
a = int(flowFile.getAttribute('a')) 

if(a == 1): 
 session.transfer(flowFile, REL_FAILURE) 
elif(a == 2): 
 session.transfer(flowFile, REL_SUCCESS) 
else: 
 session.remove(flowFile)

So the flowFiles with "a" equalling 1 or 2 are being redirected accordingly while the others are removed. You can see that SUCCESS and FAILURE relation have 1 flow file each. The third one has been deleted.

You can modify the above script according to your business logic and you should be good.

Hope that helps!

Thanks!

View solution in original post

3 REPLIES 3

@Pavan M

It might be easier to stop/delete the flow files before they get to the ExecuteScript processor using the RouteText processor.

@Pavan M You can use the remove method from the session object. Follows an example.

NiFi Flow

64662-screen-shot-2018-03-17-at-24346-am.png

Each GenerateFlowFile processor generate an empty flow with an attributed called "a" valued 1, 2 & 3 respectively.

Script in ExecuteScript processor

flowFile = session.get() 
a = int(flowFile.getAttribute('a')) 

if(a == 1): 
 session.transfer(flowFile, REL_FAILURE) 
elif(a == 2): 
 session.transfer(flowFile, REL_SUCCESS) 
else: 
 session.remove(flowFile)

So the flowFiles with "a" equalling 1 or 2 are being redirected accordingly while the others are removed. You can see that SUCCESS and FAILURE relation have 1 flow file each. The third one has been deleted.

You can modify the above script according to your business logic and you should be good.

Hope that helps!

Thanks!

Super Guru
@Pavan M

As you are not transferring any of the flowfiles to REL_FAILURE, Transfer the else flowfiles to failure relation and auto terminate the failure relation,

else:
session.transfer(flowFile, REL_FAILURE)

Auto terminate failure relation

64666-executescript.png

(or)

You can use session.remove to remove the flowfile

else:
session.remove(flowFile)

by using any of the above ways you can achieve same result as you are expecting.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.