question Re: How to skip first 10 rows of a csv and read the file in Apache Nifi in Support Questions

How to skip first 10 rows of a csv and read the file in Apache Nifi

bhadraka — Mon, 21 Aug 2023 07:49:55 GMT

I'm creating a pipeline in NiFi and I have a csv file to be extracted and I want to skip the first 10 rows and read the csv. Need some assistance on how to achieve this in Apache NiFi

Re: How to skip first 10 rows of a csv and read the file in Apache Nifi

steven-matison — Mon, 21 Aug 2023 14:16:55 GMT

@bhadraka It does not appear like this can be achieved with our CSVReader. See this JIRA looking for a feature to set the skip rows:

https://issues.apache.org/jira/browse/NIFI-8932

If your data's "10 rows" is always known and the same you could use a ReplaceText to get only 11+ rows. You would match that text, and not replace it, just take the rest.

If the first 10 rows are just ignored and of the same structure as all rows, ie not some prepending lines with headers, etc; you could also use a CSV Reader and program your flow to simply ignore the first 10 flow files.

Re: How to skip first 10 rows of a csv and read the file in Apache Nifi

bhadraka — Mon, 21 Aug 2023 15:30:12 GMT

Thanks @steven-matison for the reply. can you please explain your solution, I'm not much clear on this. my first 10 rows are known and those are always needs to be omitted and continue from 10th row. Can you explain me of how would I setup the ReplaceText processor to fulfil my requirement

Re: How to skip first 10 rows of a csv and read the file in Apache Nifi

steven-matison — Mon, 21 Aug 2023 15:37:46 GMT

Let's take this a different direction... open up a code box in your reply. Choose Preformatted:

Insert Lines 0 - 11 here

Remove anything sensitive of course.