Support Questions
Find answers, ask questions, and share your expertise

How can a create a NiFi workflow for - Forward all the files to next processor only when we get Request.json from the previous processor?

How can a create a NiFi workflow for - Forward all the files to next processor only when we get Request.json from the previous processor?

New Contributor

I am working on writing a NiFi workflow and stuck in a scenario where I have to get a few files with different formats. But all the files will be routed to next processor only when we get the json file. Thanks

3 REPLIES 3

Re: How can a create a NiFi workflow for - Forward all the files to next processor only when we get Request.json from the previous processor?

Expert Contributor

@jainN 

 

Maybe you can describe your problem a bit more?
So it maybe will be possible to give you some help.

 

f.e.
Does all files (with different formats) come from the same processor?
What is your "previous processor"?
Do you get data from InvokeHTTP?
"When we get the json file" - Do you get responses without content?

Re: How can a create a NiFi workflow for - Forward all the files to next processor only when we get Request.json from the previous processor?

New Contributor

@justenji My usecase is:

Read the files with different formats from local directory (all files from same source folder). As soon as we get a json file (else keep on waiting for a json file), forward all the files to the next processor and write them in the target folder on the basis of its format (if json format then in folder A else folder B).

 

We have to play with all the files available in source folder but only when we have *.json file in the source folder.

Re: How can a create a NiFi workflow for - Forward all the files to next processor only when we get Request.json from the previous processor?

@jainN 

 

If you are looking to route flowfiles that end in json versus those that are not, check out RouteOnAttribute with something similar to json => ${filename:endsWith('.json')}.  You would use this after using your method of choice to list/fetch the files which would then provide a $filename for every flowfile.   With this json property added to RouteOnAttribute you can drag the json route to a triggering flow, and send everything else (not json: unmatched) to a holding flow.   NiFi Wait/Notify should be able to provide the trigger logic, but there are many other ways to do it with out wait/notify by using another datastore, map cache, etc.  For example, your non json flow could simply write to a new location and finish.  Then your json flow can process that new location some known amount of time later.  The logic there is your use case ofc ourse, the point is to use RouteOnAttribute to split your flow.

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

 

Thanks,

Steven