question Creating multiple flow files and ExecuteScript error handling in Support Questions

Creating multiple flow files and ExecuteScript error handling

sean_dockery — Tue, 26 Mar 2019 08:14:12 GMT

If I am in the process of creating multiple flow files during an ExecuteScript processor, and an error occurs before I'm done, I transfer the original flow file to its failure relationship. My doubt, however, is whether or not I have to do anything with the flow files that I've created thus far.

Consider, for example, the following groovy:

def flowFile = session.get()

if (!flowFile) return

def filesDir = flowFile.getAttribute('filesDirectory')

// See https://community.hortonworks.com/questions/106878/split-one-nifi-flow-file-into-multiple-flow-file-b.html

def flowFiles = [] as List<FlowFile>

try {

// Get a list of all the trades in the portfolio directory.

File folder = new File(filesDir);

File[] listOfFiles = folder.listFiles();

for (File file : listOfFiles) {

if (file.isFile()) {

// Create a new flow file with that single trade.

def newFlowFile = session.create(flowFile)

flowFiles << newFlowFile

// put some attributes here, or do something that might throw an exception

}

catch (e) {

session.transfer(flowFile, REL_FAILURE)

return

}

session.remove(flowFile)

session.transfer(flowFiles, REL_SUCCESS)

My doubt is whether or not I need something like this in the "catch" block:

catch (e) {

// is this necessary?

for (FlowFile theFlowFile : flowFiles) {

session.remove(theFlowFile)

}

session.transfer(flowFile, REL_FAILURE)

return

}

...or will the session automatically clean them up on the basis that they haven't been directed to any relationship?

Re: Creating multiple flow files and ExecuteScript error handling

mburgess — Tue, 26 Mar 2019 20:58:27 GMT

Once a flow file has been created in a session, it must be removed or transferred before the session is committed (which happens at the end of ExecuteScript). Since your try is outside the loop that creates new flow files, you'll want to remove all the created ones, namely the flowFiles list. You can do that with simply:

session.remove(flowFiles)

rather than the loop you have in your catch statement.

Re: Creating multiple flow files and ExecuteScript error handling

sean_dockery — Wed, 27 Mar 2019 13:32:46 GMT

Thanks, Matt.

Re: Creating multiple flow files and ExecuteScript error handling

sean_dockery — Sat, 30 Mar 2019 02:15:08 GMT

@Matt Burgess

Maybe there is a spot somewhere this may be mentioned in your already-excellent ExecuteScript Cookbook articles for the next person?

Re: Creating multiple flow files and ExecuteScript error handling

mburgess — Sat, 30 Mar 2019 02:20:09 GMT

That's a great idea, thanks! I've been meaning to update it, hopefully sooner than later 🙂