Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

NiFi: FetchFile processor appends new line at end

avatar
Expert Contributor

When you retrieve the content of a file through above processor, in the flowfile we can see a new line character appended at the end. Is this an expected behavior? Any reason for this behavior and how it can be avoided?

1 ACCEPTED SOLUTION

avatar
Rising Star
@Kumar

The NiFi provenance UI adds a newline to your content. If you go look at the raw file being fetched or moved through the FetchFile processor you will notice that there actually isn't a new line appended. If I use the same input you do, my provenance UI will show a new line (line 2) after the text "Line1", identical to your screenshot.

However if i look at where the "move" content of the fetchFile Processor is placed, i see that there isn't a newline. Also if i do a putFile i notice there isn't a newline either. Below are some screenshots to show:

Raw file:

10470-raw.jpg

Provenance shows a new line (line 2):

10471-provenance.jpg

The output of Putfile:

10472-putfileoutput.jpg

The output of having MoveFile in the fetchFile processor:

10473-fetchfilemoveoutput.jpg

The overall flow used to test:

10474-screen-shot-2016-12-18-at-85911-pm.png

View solution in original post

4 REPLIES 4

avatar
Guru

FetchFile does not append a new line character if it is not present in the original file.

When I run ListFiles -> FetchFile -> PutFile and view in text editor, the file content is identical among the following files (including new line characters viewable via the text editor):

  • original file on disk
  • file contents downloaded from FetchFile (Provenance/Contents)
  • file contents downloaded from PutFile (Provenance/Contents)
  • file put to disk

If you are not getting this behavior, please provide details.

avatar
Expert Contributor

Sample file:

10468-sample.jpg

NiFi flow:

10467-nifi.jpg

After Fetchfile (flowfile content):

10469-after-fetch-file.jpg

avatar
Guru

@Kumar, as tests by me and @Devin Pinkston show, it is the actual file content that you need to look at (not the UI). Thus ... no fears of the processor adding a new line.

avatar
Rising Star
@Kumar

The NiFi provenance UI adds a newline to your content. If you go look at the raw file being fetched or moved through the FetchFile processor you will notice that there actually isn't a new line appended. If I use the same input you do, my provenance UI will show a new line (line 2) after the text "Line1", identical to your screenshot.

However if i look at where the "move" content of the fetchFile Processor is placed, i see that there isn't a newline. Also if i do a putFile i notice there isn't a newline either. Below are some screenshots to show:

Raw file:

10470-raw.jpg

Provenance shows a new line (line 2):

10471-provenance.jpg

The output of Putfile:

10472-putfileoutput.jpg

The output of having MoveFile in the fetchFile processor:

10473-fetchfilemoveoutput.jpg

The overall flow used to test:

10474-screen-shot-2016-12-18-at-85911-pm.png