question Re: Deleting specific lines of an csv file using Apache NIFI in Archives of Support Questions (Read Only)

Deleting specific lines of an csv file using Apache NIFI

onestoneh — Sun, 18 Aug 2019 06:23:55 GMT

Hi,

I ingested csv files using ListFile and FetchFile processors.

Those files contain dates of 2016 in it which should all be deleted before saving it in cassandra.

The table looks like this:

I got a problem with extracting/deleting these lines:

It looks like this:

with following properties in RouteText (testing it out with different options)

It is still sending all lines to the next processor. (or none)

Do you have an idea of an solution?

Thanks in advance!

best regards

Re: Deleting specific lines of an csv file using Apache NIFI

Shu_ashu — Sun, 18 Aug 2019 06:23:20 GMT

@Salda Murrah

if you want to filter out 2016 and 2017 records then in route on content processor change the below properties

Routeoncontent with Contains as Matching Strategy:-

Keep Matching Strategy as Contains

Route Strategy as Route to each matching Property Name

add new properties

1.2016 as 2016 //check for content if it contains 2016 then route to this relation
2.2017 as 2017 //check for content if it contains 2017 then route to this relation

Routeoncontent configs:-

(or)

Routeoncontent with RegularExpression as Matching Strategy:-

If you want to check the contents of flow file with regular expressions then

Keep Matching Strategy as Matches Regular Expression

Route Strategy as Route to each matching Property Name

add new properties

1.2016 as ^.*;.*2016.*;.*$  //check for content if it contains 2016 then route to this relation
2. 2017 as ^.*;.*2017.*;.*$ //check for content if it contains 2017 then route to this relation

Routeoncontent with RegularExpression Config:-

Flow:-

ListFile --> FetchFile --> SplitText //split into 1 line --> RouteonContent //you can use either Contains (or) matches regular expression As Matching Strategies --> .... -->PutCassandraQL

Re: Deleting specific lines of an csv file using Apache NIFI

onestoneh — Wed, 15 Nov 2017 23:22:53 GMT

Thank you very much! It worked very well for me!