Remove first few lines in a text/csv flowfile content in Apache NiFi

Hi @glad1 ,

Can you elaborate more on the data that you want to remove? For example if the data is part of the CSV and it has unique value in one or more columns, then you can use QueryRecord processor where the query exclude records with this unique value. If the data is out of the CSV - like a header information - then depending how this data look like and if its surrounded with some special characters then you can use ReplaceText Processor with regex that would isolate those lines and then replace them with empty space and so on. If you can provide some sample data it would help in figuring out the best solution for this scenario.

Thanks

I you're confident the data returned is consistent and always more than 7 lines...then a quick and dirty would be a Groovy script like this.

import java.nio.charset.StandardCharsets

FlowFile flowFile = session.get()
if(!flowFile) return

flowFile = session.write(flowFile, {inputStream, outputStream ->
                                      String[] data = inputStream.readLines()
                                      data = data.drop(7)
                                      outputStream.write(data.join("\n").getBytes(StandardCharsets.UTF_8))
                                  } as StreamCallback)
session.transfer(flowFile, REL_SUCCESS)