Dynamic parameter for GetFile/ListFile processor - NiFi

[#1]

My workflow is like below.

ListenHTTP(i get a directory name here) --> SplitText --> ExtractText(directory name added as attribute)

Now after this i will have to use that attribute directoryname and extract all the files in that local dir and put that into HDFS. I understand GetFile/ListFile could do this, but how do we provide a dynamic directory name to that processor?

[#2]

@Vignesh I

Using the ListFile processor as an example, this is how you would use the attribute directoryname to specify the directory to list files

Then you would pass the list of files to a FetchFile processor.

[#3]

But how do I provide the parameter directoryname. As far as I understand GetFile doesn't accept an incoming flow.

[#4]

@Vignesh I

You won't be using GetFile, FetchFile will replace it. Use ListFile to create the list of files and then pass the list of files to a FetchFile processor, it will pick up the files. If you look at the FetchFile processor, it will use the attributes from the list of files to pull the files.

If you are running a cluster, make sure to configure the ListFile processor to run on the primary node only.

[#5]

@Vignesh I

Did this resolve your issue?

[#6]

I did. Used a executescript processor in which I will pass the directoryname as a dynamic parameter which I get from the listenHTTP and after the executescript I used fetchfile to read a single file and then used puthdfs to load into HDFS.