Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi: How to connect to a GetFile Processor?

Solved Go to solution
Highlighted

NiFi: How to connect to a GetFile Processor?

Super Collaborator

I have a flow that uses an ExecuteStreamCommand processor to populate a folder with some data.

The actual flow files are essentially just arguments to a script that eventually generates some output.

When the script is done, I want to go pick up the output using GetFile or FetchFile, and then continue some further processing.

However, I can't seem to connect my ExecuteStreamCommand processor to a GetFile or FetchFile processor. The GUI simply won't allow it.

Can anyone straighten me out here?

Is there some reason that what I'm trying to do is invalid?

Is there a better way to accomplish this?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi: How to connect to a GetFile Processor?

Super Guru

You could use ListFile in a separate flow, it will keep track of the files it has listed so far, such that if your ExecuteStreamCommand generates more files in the specified location(s), only the new files will be listed the next time ListFile runs. Then ListFile can be routed to FetchFile to get the contents of the new files, etc.

View solution in original post

2 REPLIES 2
Highlighted

Re: NiFi: How to connect to a GetFile Processor?

Super Guru

You could use ListFile in a separate flow, it will keep track of the files it has listed so far, such that if your ExecuteStreamCommand generates more files in the specified location(s), only the new files will be listed the next time ListFile runs. Then ListFile can be routed to FetchFile to get the contents of the new files, etc.

View solution in original post

Highlighted

Re: NiFi: How to connect to a GetFile Processor?

Super Collaborator

Thanks.

I landed at a similar solution using GetFile.

Don't have an account?
Coming from Hortonworks? Activate your account here