Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi: FetchFile processor appends new line at end

Solved Go to solution

NiFi: FetchFile processor appends new line at end

Rising Star

When you retrieve the content of a file through above processor, in the flowfile we can see a new line character appended at the end. Is this an expected behavior? Any reason for this behavior and how it can be avoided?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: NiFi: FetchFile processor appends new line at end

Contributor
@Kumar

The NiFi provenance UI adds a newline to your content. If you go look at the raw file being fetched or moved through the FetchFile processor you will notice that there actually isn't a new line appended. If I use the same input you do, my provenance UI will show a new line (line 2) after the text "Line1", identical to your screenshot.

However if i look at where the "move" content of the fetchFile Processor is placed, i see that there isn't a newline. Also if i do a putFile i notice there isn't a newline either. Below are some screenshots to show:

Raw file:

10470-raw.jpg

Provenance shows a new line (line 2):

10471-provenance.jpg

The output of Putfile:

10472-putfileoutput.jpg

The output of having MoveFile in the fetchFile processor:

10473-fetchfilemoveoutput.jpg

The overall flow used to test:

10474-screen-shot-2016-12-18-at-85911-pm.png

4 REPLIES 4

Re: NiFi: FetchFile processor appends new line at end

Guru

FetchFile does not append a new line character if it is not present in the original file.

When I run ListFiles -> FetchFile -> PutFile and view in text editor, the file content is identical among the following files (including new line characters viewable via the text editor):

  • original file on disk
  • file contents downloaded from FetchFile (Provenance/Contents)
  • file contents downloaded from PutFile (Provenance/Contents)
  • file put to disk

If you are not getting this behavior, please provide details.

Highlighted

Re: NiFi: FetchFile processor appends new line at end

Rising Star

Sample file:

10468-sample.jpg

NiFi flow:

10467-nifi.jpg

After Fetchfile (flowfile content):

10469-after-fetch-file.jpg

Re: NiFi: FetchFile processor appends new line at end

Guru

@Kumar, as tests by me and @Devin Pinkston show, it is the actual file content that you need to look at (not the UI). Thus ... no fears of the processor adding a new line.

Re: NiFi: FetchFile processor appends new line at end

Contributor
@Kumar

The NiFi provenance UI adds a newline to your content. If you go look at the raw file being fetched or moved through the FetchFile processor you will notice that there actually isn't a new line appended. If I use the same input you do, my provenance UI will show a new line (line 2) after the text "Line1", identical to your screenshot.

However if i look at where the "move" content of the fetchFile Processor is placed, i see that there isn't a newline. Also if i do a putFile i notice there isn't a newline either. Below are some screenshots to show:

Raw file:

10470-raw.jpg

Provenance shows a new line (line 2):

10471-provenance.jpg

The output of Putfile:

10472-putfileoutput.jpg

The output of having MoveFile in the fetchFile processor:

10473-fetchfilemoveoutput.jpg

The overall flow used to test:

10474-screen-shot-2016-12-18-at-85911-pm.png