- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
NiFi: FetchFile processor appends new line at end
- Labels:
-
Apache NiFi
Created ‎12-18-2016 09:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
When you retrieve the content of a file through above processor, in the flowfile we can see a new line character appended at the end. Is this an expected behavior? Any reason for this behavior and how it can be avoided?
Created on ‎12-19-2016 02:01 AM - edited ‎08-18-2019 05:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The NiFi provenance UI adds a newline to your content. If you go look at the raw file being fetched or moved through the FetchFile processor you will notice that there actually isn't a new line appended. If I use the same input you do, my provenance UI will show a new line (line 2) after the text "Line1", identical to your screenshot.
However if i look at where the "move" content of the fetchFile Processor is placed, i see that there isn't a newline. Also if i do a putFile i notice there isn't a newline either. Below are some screenshots to show:
Raw file:
Provenance shows a new line (line 2):
The output of Putfile:
The output of having MoveFile in the fetchFile processor:
The overall flow used to test:
Created ‎12-18-2016 05:36 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
FetchFile does not append a new line character if it is not present in the original file.
When I run ListFiles -> FetchFile -> PutFile and view in text editor, the file content is identical among the following files (including new line characters viewable via the text editor):
- original file on disk
- file contents downloaded from FetchFile (Provenance/Contents)
- file contents downloaded from PutFile (Provenance/Contents)
- file put to disk
If you are not getting this behavior, please provide details.
Created on ‎12-18-2016 11:41 PM - edited ‎08-18-2019 05:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sample file:
NiFi flow:
After Fetchfile (flowfile content):
Created ‎12-19-2016 12:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Kumar, as tests by me and @Devin Pinkston show, it is the actual file content that you need to look at (not the UI). Thus ... no fears of the processor adding a new line.
Created on ‎12-19-2016 02:01 AM - edited ‎08-18-2019 05:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The NiFi provenance UI adds a newline to your content. If you go look at the raw file being fetched or moved through the FetchFile processor you will notice that there actually isn't a new line appended. If I use the same input you do, my provenance UI will show a new line (line 2) after the text "Line1", identical to your screenshot.
However if i look at where the "move" content of the fetchFile Processor is placed, i see that there isn't a newline. Also if i do a putFile i notice there isn't a newline either. Below are some screenshots to show:
Raw file:
Provenance shows a new line (line 2):
The output of Putfile:
The output of having MoveFile in the fetchFile processor:
The overall flow used to test:
