- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
NIFI - remove header and footer lines from CSV
- Labels:
-
Apache NiFi
Created on
‎02-18-2020
12:38 PM
- last edited on
‎02-18-2020
03:07 PM
by
ask_bill_brooks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am attempting to remove multiple header and footer/trailer lines from a CSV. The CSVReader will only allow me to skip the first line, which isn't enough,and I don't see any way to skip trailer records. Is my only option to use an ExternalScript processor and handle it outside of nifi? Like a shell script with head and tail?
Created ‎02-18-2020 04:45 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@JohnYaya if you can show a sample, it would be very helpful.
If your header and footer is static, and always predictable for a specific file you can replaceText and get "in between" lines using those static header/footer matches with a very creative regex...
Created ‎02-18-2020 08:39 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well, I'm trying to put together a fairly generic service to ingest files. One of the reqs is to be able to strip off any header or trailer lines. Files can come from any number of sources, so I don't think there will be any kind of pattern for a regex.
