Ignore first line of a file and process second line onwards in NiFi

AlokKumar — Wed, 28 Aug 2024 15:39:29 GMT

Say I have a file having 100 records. In this file I want to ignore the first line as it's data I do not want to process (It contains some info like file name, date and number of records). I want to process from second line. How to do that?

Re: Ignore first line of a file and process second line onwards in NiFi

MattWho — Mon, 09 Sep 2024 20:07:20 GMT

@AlokKumar

The more detail you provide the better response you will get.
In this case, more detail around the content/format of the source file (samples are great) and how it is being or will be processed downstream.

A NiFi FlowFile is what is passed from one processor component to the next. There is no direct relationship between what processing was done in one processor versus another. Each Processor will read the FlowFile (consists of FlowFile Attributes/metadata and FlowFile content) and execute its processor code against the FlowFile and output a FlowFile(s) to one or more outbound relationships. So processing against the content of A FlowFile becomes the responsibility of each processor separately. So the questions asked correlate to the answer you'll get.

With what you shared, I can only suggest that you remove the first line from the content of the FlowFile so that downstream processor will only have line 2 to process against.
This can be accomplished easily with a ReplaceText processor. Simply configure it for "Evaluation Mode" = "line-by-line", "line-by-line Evaluation Mode" = "First-Line", and in "Replacement Value" property "Set empty string" check box.

The output FlowFile' content, which is sent to the success relationship, will have everything except its first line.

Do you need to preserve line 1?
If so, maybe use ExtractText to extract line 1 to a FlowFile attribute you can then use after processing to add back to Content again using ReplaceText.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

question Re: Ignore first line of a file and process second line onwards in NiFi in Support Questions

Ignore first line of a file and process second line onwards in NiFi

Re: Ignore first line of a file and process second line onwards in NiFi