Skip to main content
sean_dockery
Silver ⭐️⭐️
March 26, 2019
Solved

Creating multiple flow files and ExecuteScript error handling

If I am in the process of creating multiple flow files during an ExecuteScript processor, and an error occurs before I'm done, I transfer the original flow file to its failure relationship. My doubt, however, is whether or not I have to do anything with the flow files that I've created thus far.

Consider, for example, the following groovy:


def flowFile = session.get()

if (!flowFile) return


def filesDir = flowFile.getAttribute('filesDirectory')


// See https://community.hortonworks.com/questions/106878/split-one-nifi-flow-file-into-multiple-flow-file-b.html

def flowFiles = [] as List<FlowFile>


try {

// Get a list of all the trades in the portfolio directory.

File folder = new File(filesDir);

File[] listOfFiles = folder.listFiles();

for (File file : listOfFiles) {

if (file.isFile()) {

// Create a new flow file with that single trade.


def newFlowFile = session.create(flowFile)

flowFiles << newFlowFile

// put some attributes here, or do something that might throw an exception

}

}

}

catch (e) {

session.transfer(flowFile, REL_FAILURE)

return

}


session.remove(flowFile)

session.transfer(flowFiles, REL_SUCCESS)




My doubt is whether or not I need something like this in the "catch" block:


catch (e) {

// is this necessary?

for (FlowFile theFlowFile : flowFiles) {

session.remove(theFlowFile)

}


session.transfer(flowFile, REL_FAILURE)

return

}


...or will the session automatically clean them up on the basis that they haven't been directed to any relationship?


mburgess
mburgessAnswer
Gold ⭐️⭐️⭐️
March 26, 2019

Once a flow file has been created in a session, it must be removed or transferred before the session is committed (which happens at the end of ExecuteScript). Since your try is outside the loop that creates new flow files, you'll want to remove all the created ones, namely the flowFiles list. You can do that with simply:

session.remove(flowFiles)

rather than the loop you have in your catch statement.

4 replies

mburgess
Gold ⭐️⭐️⭐️
mburgessAnswer
Gold ⭐️⭐️⭐️
March 26, 2019

Once a flow file has been created in a session, it must be removed or transferred before the session is committed (which happens at the end of ExecuteScript). Since your try is outside the loop that creates new flow files, you'll want to remove all the created ones, namely the flowFiles list. You can do that with simply:

session.remove(flowFiles)

rather than the loop you have in your catch statement.

Translate
sean_dockery
Silver ⭐️⭐️
March 27, 2019

Thanks, Matt.

Translate