NIFI ListS3 doesn't take input

[#1]

Hi,

I am using the ListS3 process in NIFI. But I found out it doesn't accept input.

Does that mean I have to use static Prefix? I couldn't find a way to pass upstream flowfile attribute into ListS3 to be used as Prefix.

I was thinking about modify the source code and create a new custom process to do List with variables from upstream flowfile, but the code is too hard for me to understand.

Is anyone know an easy way to do it?

Thanks very much

[#2]

@Wei Wu

ListS3 Processor doesn't allow any upstream connections if you want to fetch some files from the S3 object without listS3 processor then you need to keep your filenames as attributes to the flowfile then use FetchS3 object processor to fetch the files from s3 bucket.

To get all the filenames from the S3 bucket you need to use some kind of RestApi call or shell script to list out all filenames from the Bucket then extract the filenames as attributes to the flowfile.

Then feed the connection to the FetchS3 processor to fetch actual file contents without ListS3 processor.

Refer to this link to get more context over these kind of usecase.

[#3]

I understand that it'd be pointless to send a FlowFile to the ListS3 processor, but it still seems like we should be able to attach it to a Wait processor and have it trigger that way.