Support Questions

Find answers, ask questions, and share your expertise
Announcements
Now Live: Explore expert insights and technical deep dives on the new Cloudera Community BlogsRead the Announcement

NIFI Python filtering JSON

avatar
Explorer

I'm working on a custom Python script to execute inside 'ExecuteScript' processor in NIFI.

 

This processor receives input JSON flowfiles and only those that are present in variable filter inside script, should pass through. So the idea is filter JSON based on one column.

 

My current script looks like this:

 

 

import json

flowFile = session.get()

if flowFile is not None:
    filter = ['1000007', '1000008'] 
    
    json_content = json.loads(flowFile.read().decode('utf-8'))
    if json_content['Customer_AccountID'] in patata:
        filtered_content = json.dumps(json_content)
        flowFile.write(bytearray(filtered_content, 'utf-8'))
        session.transfer(flowFile, REL_SUCCESS)
    else:
        session.remove(flowFile)
        session.transfer(flowFile, REL_FAILURE)

 

 

For now is not filtering anything, no output at all. Any idea why?

anton123_0-1679041028615.png

 

1 REPLY 1

avatar

@anton123 Please check these references, there is important information here that everyone needs to master in order to work with ExecuteScript:

https://community.hortonworks.com/articles/75032/executescript-cookbook-part-1.html

https://community.hortonworks.com/articles/75545/executescript-cookbook-part-2.html

https://community.hortonworks.com/articles/77739/executescript-cookbook-part-3.html

 

Some suggestions:

  1. Make sure you have all the imports you need
  2. Make sure you are using the correct variable names for "filter".  You have this defined but its not used.