Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI Execute script- Node disconnected

avatar
Explorer

Hi,

I am facing a problem with my Execute script,

My python script generate a FlowFile with an attribute :

flowFile = session.get() 
try:
    if (flowFile != None):
                
            Table_name =  flowFile.getAttribute('Table_Name')
            Table_path =   flowFile.getAttribute('Table_Path')
            Table_schema = flowFile.getAttribute('Table_Schema')

            Table_path = Table_path.split(',')
            Table_schema = Table_schema.split(',')

            project_query = '{'
            collection = Table_path[0]
            father = Table_path[-1]
            for table in Table_path:
                project_query = project_query + r'"{}._id" : 1, '.format(table)


            for column in Table_schema:
                project_query = project_query + r'"{}.{}.{}" : 1, '.format(father,Table_name,column)

            project_query = project_query[:-2]

            project_query = project_query + '}'

            flowFile = session.putAttribute(flowFile, 'project_query', project_query)
            flowFile = session.putAttribute(flowFile, 'Collection', collection)
            session.transfer(flowFile, REL_SUCCESS)

except Exception as e:
    flowFile = session.putAttribute(flowFile, 'ERROR', e)
    session.transfer(flowFile, REL_FAILURE)

This scripts caused my node to disconnect,

any advice ?

 

6 REPLIES 6

avatar
Super Guru

@Asim- ,

 

Which version of NiFi are you using? I tested this on NiFi 1.15.2 and there was no "disconnection", although for some tests no output was being generated even thought there were no errors.

 

The only thing that I could find was the following line:

    flowFile = session.putAttribute(flowFile, 'ERROR', e)

 

The variable "e" contains an Exception object, not a string, and putAttribute only accepts string. That line would cause an exception when executed, which could explain the issue that you're seeing.

 

Try changing it to the following and see if it helps:

    flowFile = session.putAttribute(flowFile, 'ERROR', str(e))

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

This screen shot of my Data,  

is there something that cause this issue?

also should I commit session in the scripts?

many thanks.Capture2.PNG

avatar
Explorer

also how NIFI work with Scripts, in field of threads?

avatar
Super Guru

Have you tried the change that I suggested?

Which version of NiFi do you have?

Can you share the attributes of one of your flow files?

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

Which version of NiFi do you have? Nifi version : 1.15.2

Have you tried the change that I suggested? : There is not Failure FlowFile so I don't think its cause this isssue

 

( sorry I don't mentioned that, the NIFI has two nodes and the secondary node got disconnect in some time of the run of the Flow, not always, so the issue cause the disconnect the sync between two node, is this caused by thread consuming?)

 

Can you share the attributes of one of your flow files? : att.PNG

avatar
Super Guru

@Asim- ,

 

There is not Failure FlowFile so I don't think its cause this isssue

 

It doesn't hurt making the modification and trying. What I noticed is that if there's any exception during the execution of your code, due to the problem in that line that I mentioned, no error is shown in the UI and no flowfiles go to either the success or failure relationships.

 

Please make the change and test to see if something else appears.

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.