question Re: NiFi routeoncontent processor in Archives of Support Questions (Read Only)

NiFi routeoncontent processor

edmund_prout — Tue, 02 Aug 2016 04:20:53 GMT

I'm building a nifi flow with the nifi GUI. As part of the flow I have series of flat files I'm ingesting, which contains lines that I don't want in my data flow. These lines all start with the hash/pound symbol #. Any ideas how to filter these lines out? I was thinking a routeoncontent processor, but I'm not sure how to make it filter out lines.

Re: NiFi routeoncontent processor

sunile_manjee — Tue, 02 Aug 2016 04:25:48 GMT

@Ed Prout Take a look at the extractText processor here

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExtractText/index.html

How to use extract processor example is here

https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates?preview=/57904847/61320129/CsvToJSON.xml

Additional here is another example to extract text using NiFi

https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates?preview=/57904847/59211793/simple-httpget-route.template.xml

Re: NiFi routeoncontent processor

cstanca — Tue, 02 Aug 2016 04:26:15 GMT

@Ed Prout

Split that text file by line using the SplitText processor first: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.SplitText/index.html

Then you can exclude the lines you don't using a basic regex.

Good example here: https://github.com/xmlking/nifi-examples/tree/master/split-route

Re: NiFi routeoncontent processor

pvillard — Tue, 02 Aug 2016 04:34:16 GMT

Hi @Ed Prout,

If you don't care about having one FlowFile by line from your input file, I'd suggest you to use RouteText processor with a matching strategy 'starts with' and adding a custom property like 'prefix' with the value '#'. This will create a relationship 'prefix' with all lines starting with # (then you'll want to route the lines for relationship 'unmatched' if you want lines not starting with a #).

Hope this helps.

Re: NiFi routeoncontent processor

edmund_prout — Wed, 03 Aug 2016 03:54:00 GMT

I went with your answer first Pierre, and it worked. Thanks!