Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

NiFi: How to connect to a GetFile Processor?

Super Collaborator

I have a flow that uses an ExecuteStreamCommand processor to populate a folder with some data.

The actual flow files are essentially just arguments to a script that eventually generates some output.

When the script is done, I want to go pick up the output using GetFile or FetchFile, and then continue some further processing.

However, I can't seem to connect my ExecuteStreamCommand processor to a GetFile or FetchFile processor. The GUI simply won't allow it.

Can anyone straighten me out here?

Is there some reason that what I'm trying to do is invalid?

Is there a better way to accomplish this?

1 ACCEPTED SOLUTION

Super Guru

You could use ListFile in a separate flow, it will keep track of the files it has listed so far, such that if your ExecuteStreamCommand generates more files in the specified location(s), only the new files will be listed the next time ListFile runs. Then ListFile can be routed to FetchFile to get the contents of the new files, etc.

View solution in original post

2 REPLIES 2

Super Guru

You could use ListFile in a separate flow, it will keep track of the files it has listed so far, such that if your ExecuteStreamCommand generates more files in the specified location(s), only the new files will be listed the next time ListFile runs. Then ListFile can be routed to FetchFile to get the contents of the new files, etc.

Super Collaborator

Thanks.

I landed at a similar solution using GetFile.