Created on 10-12-2019 06:58 AM - edited 10-12-2019 07:34 AM
How to route fixed width records into multiple Routes to use individual copybooks,
I used earlier Route on content processor if my data starts as below and its working as expected.
Example Data
00ABD0000000100000103200A08LL30003 0000000000220700840940
10ABD0000000210000100060A08LL30003 0000000000220700840940
In nifi we used earlier
recordtype01=00
recordtype02=10
route01 =(${recordtype01}.*)
route02 =(${recordtype02}.*)
Based on above regex starts with 00 records will go to one route and 10 records will go to different
New requirement
But now we are getting as below for other feed, so to determine which copybook to use for each row on the file, We would need to look at position 12:2 and route based on the value
for example if 12,2 =00 then those records should go to route01 ,if 12,2=10 then those records should go to route02
Sample content of flow file
ABD0000000100000103200A08LL30003 0000000000220700840940
ABD0000000210000100060A08LL30003 0000000000220700840940
any idea how to Route on content of flow file to different routes to add required copy book or any thoughts with different processors in nifi also helpful
Thanks in advance
Created on 10-12-2019 07:17 PM - edited 10-12-2019 07:18 PM
one way would be using Route Text processor to get first field value until first space.. and then apply NiFi expression language substring function on the extracted attribute value to check/route the line based on the value..
Regex to capture first field value until first space..
([^ ]+)(.*)
Another method:
As you are having fixed width file so by using ReplaceText Processor we can create delimited file and by using QueryRecord processor add new SQL query to check
substring(firstfield_name,Start,endposition)="00"
Add two new queries in QueryRecord processor to check substring value and route the records dynamically based on matched criteria.
-
If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.
Created on 10-18-2019 12:08 PM - edited 10-18-2019 02:09 PM
Please help me on this issue as our work got Hold due to not able to split the file into multiple files
Option -1: I tried to use Route Text processor to get first field value until first space, But in file as i have millions of records, i need to get each group into individual flow
I have record types of 25 categories , so for example in my file if i have 100 records for record type =00 then those all 100 should go to route1 and if i have 250 records for record type =10 then those all 250 should go to route2 ...... etc
If i use route text processor , how many i need to write routes ?
For example i am writing as below ,its not giving my expected results
Route01 = ([^ ]+)(.*)
Sample content of flow file
ABD0000000100000103200A08LL30003 0000000000220700840940
ABD0000000210000100060A08LL30003 0000000000220700840940
for every record as seen above i have space after 30003 to next value
I am not getting as expected based on below regex.Could you please let me know how i can i do this one
Option 2. I also tried to use Extract Text Processor to extract first field value until first space as i have l multiple record types i am not getting all records extracted into my attribute to route in next processor
once we are able to get sub string from file content then we want to assign individual sachems as below as each record type has different schema
Please suggest us if anything wrong i am doing as i am new to nifi
Created 10-22-2019 11:04 AM
Any help on this is a great help for us as we are unable to route the records based on substring value on the flow file content