I have create a data pipeline that fetches an excel file and copies it into a table in a SQL Server. I have used ExtractText processor to filter rows by using a regular expression, as from the post at https://community.cloudera.com/t5/Community-Articles/NiFi-ETL-Removing-columns-filtering-rows-changi...
I have validated the used regular expression at https://www.freeformatter.com/java-regex-tester.html#ad-output
However, the filter produces no match and no data is written in the SQL Server.
By omitting the filter, the flow works fine so the problem is clearly in the ExtractText processor.
I have searched for possible hints in this and other forums with no success. Any idea what might be wrong?
Below is my complete configuration, the used source data and the configured SQL table.
Thanks a lot in advance!
Flow:
GetFile processor:
ConvertExcelToCSVProcessor:
SplitText processor:
ExtractText processor:
PutDatabaseRecord processor:
DBCPConnectionPool controller service:
CSVReader controller service:
Source data
Target table:
I have managed to solve the issue by using the RouteText processor.
I have added a propriety containing a regular expression that matches the rows I want to omit and I have configured the corresponding relationship to be terminated locally.
Still, it would be nice to know how to implement the same functionality by using the ExtractText processor.
