Created 08-21-2023 12:49 AM
I'm creating a pipeline in NiFi and I have a csv file to be extracted and I want to skip the first 10 rows and read the csv. Need some assistance on how to achieve this in Apache NiFi
Created 08-21-2023 07:16 AM
@bhadraka It does not appear like this can be achieved with our CSVReader. See this JIRA looking for a feature to set the skip rows:
https://issues.apache.org/jira/browse/NIFI-8932
If your data's "10 rows" is always known and the same you could use a ReplaceText to get only 11+ rows. You would match that text, and not replace it, just take the rest.
If the first 10 rows are just ignored and of the same structure as all rows, ie not some prepending lines with headers, etc; you could also use a CSV Reader and program your flow to simply ignore the first 10 flow files.
Created 08-21-2023 08:30 AM
Thanks @steven-matison for the reply. can you please explain your solution, I'm not much clear on this. my first 10 rows are known and those are always needs to be omitted and continue from 10th row. Can you explain me of how would I setup the ReplaceText processor to fulfil my requirement
Created 08-21-2023 08:37 AM
Let's take this a different direction... open up a code box in your reply. Choose Preformatted:
Insert Lines 0 - 11 here
Remove anything sensitive of course.