I am maintaining a log of processed serial numbers in a file on disk. Now if I run the flow again I needed to match the serial number attribute on each NiFi flowfile with the list of serial numbers stored in a file on disk. Case: 1)I am fetching the file containing a list of serial numbers on disk using FetchFile processor. 2)Using routeonContent processor to route any serial number not present in the file further by searching for the serial number attribute in the file. 3) I auto terminate the matched ones since its already processed. This Procedure needs me to read the file containing the list of serial numbers for every flowflie which in turn causes the memory to bloat. Is there a way to read the disk file only once and match all flow file attributes to the same "static" instance of the disk file? ... View more

I wanted to use NiFi's posthttp/invokeHttp processor to post a pdf to an API. But considering the following curl request to replicate in NiFi: curl -X POST "http://ipaddress:port/api/" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "pdf_file=@sample.pdf;type=application/pdf" Which property takes the -F information in nifi attributes. ... View more

NiFi doesn't seem to take -F as an input while invoking the following request: curl -X POST "http://ipaddress:port/api/" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "pdf_file=@sample.pdf;type=application/pdf" It seems to me that it is by default requesting in this format: curl -X POST "http://ipaddress:port/api/" -H "accept: application/json" -H "Content-Type: multipart/form-data" -d "pdf_file=@sample.pdf;type=application/pdf" Sending raw data instead of form data. The API hence responds with error saying: "400 Bad Request: The browser (or proxy) sent a request that this server could not understand." Whereas Using PostHttp: setting the same URL and content-type as application/form-data gives me 500 internal server error server logs: readv() failed (104: Connection reset by peer) while reading upstream ... View more

I am able to get the details of the prefix I need to recursively list only after execution of four initial processors. Since I can not use ListS3 processor in the middle of the flow (It does not take an incoming relationship). How can I list the prefix in S3 recursively. I fetch a json file from S3 bucket that contains the prefix information. This prefix changes daily. Then I need to list the prefix recursively. aws s3 ls s3://{Bucket Name}/{prefix}/ --recursive ... View more

I have a kinesis stream that I need to consume in to my nifi flow. Going through the processors I can find a PutKinesisStream processor but no such thing as GetKinesisStream. ... View more

I use JSON Paths as attributes to nifi flowfiles (of JSON mime type). I want to evaluate these JSON Path over the content of the flowfiles but I can't use Evaluate JSON path processor as it does not support expression language. How can I do this as I can't hard code the JSON Paths in the processor, they are varying file to file. ... View more

I was using UnpackContent processor to unpack tar files decompressed from *.gz files. These files contain compressed tex files. It becomes critical for me to preserve the folder structure related information to later use them to parse the tex files properly as the tex files call each other with the folder prefixes. The unpack Content processor doesn't give me any details about it in the attributes or content. Lets take an example of data.tex that is packed in a subfolder "child" in a parent file 8808.gz. This file is kept in s3 on a prefix S3://PRE/DIST/. When I unpack I get: absolute.path=/usr/local/Cellar/nifi/1.1.1/libexec/ path=null/ as nifi attributes Am I not using the Unpack processor properly? ... View more