Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi routeoncontent processor

avatar
Contributor

I'm building a nifi flow with the nifi GUI. As part of the flow I have series of flat files I'm ingesting, which contains lines that I don't want in my data flow. These lines all start with the hash/pound symbol #. Any ideas how to filter these lines out? I was thinking a routeoncontent processor, but I'm not sure how to make it filter out lines.

1 ACCEPTED SOLUTION

avatar

Hi @Ed Prout,

If you don't care about having one FlowFile by line from your input file, I'd suggest you to use RouteText processor with a matching strategy 'starts with' and adding a custom property like 'prefix' with the value '#'. This will create a relationship 'prefix' with all lines starting with # (then you'll want to route the lines for relationship 'unmatched' if you want lines not starting with a #).

Hope this helps.

View solution in original post

4 REPLIES 4

avatar
Super Guru

@Ed Prout

Split that text file by line using the SplitText processor first: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.SplitText/inde...

Then you can exclude the lines you don't using a basic regex.

Good example here: https://github.com/xmlking/nifi-examples/tree/master/split-route

avatar

Hi @Ed Prout,

If you don't care about having one FlowFile by line from your input file, I'd suggest you to use RouteText processor with a matching strategy 'starts with' and adding a custom property like 'prefix' with the value '#'. This will create a relationship 'prefix' with all lines starting with # (then you'll want to route the lines for relationship 'unmatched' if you want lines not starting with a #).

Hope this helps.

avatar
Contributor

I went with your answer first Pierre, and it worked. Thanks!