How to add the parent row values to child row in the NIFI flow file?

kum — Thu, 23 May 2024 08:12:29 GMT

Example:
Flowfile data

branches,name,id,acc,serialNo

parent1,abc,123,1234567788,1

child1,eee,345

child2,rrr,678

parent2,mno,786,97865662762,2

child1,ttt,764

child2,ryy,456
output:

parent1,abc,123,1234567788,1

child1,eee,345,1234567788,1

child2,rrr,678,1234567788,1

parent2,mno,786,97865662762,2

child1,ttt,764,97865662762,2

child2,ryy,456,97865662762,2
I have to add the parent last two columns details to Childs of the same parent like parent1 last column details to child1, child2 rows in the flow file.

Please suggest a possible way to use the processor.

Re: How to add the parent row values to child row in the NIFI flow file?

angela683h — Thu, 23 May 2024 10:42:15 GMT

@kum dollar tree compass wrote:
Example:
Flowfile data
branches,name,id,acc,serialNo
parent1,abc,123,1234567788,1
child1,eee,345
child2,rrr,678
parent2,mno,786,97865662762,2
child1,ttt,764
child2,ryy,456
output:
parent1,abc,123,1234567788,1
child1,eee,345,1234567788,1
child2,rrr,678,1234567788,1
parent2,mno,786,97865662762,2
child1,ttt,764,97865662762,2
child2,ryy,456,97865662762,2
I have to add the parent last two columns details to Childs of the same parent like parent1 last column details to child1, child2 rows in the flow file.
Please suggest a possible way to use the processor.

Hello, @kum

You can achieve this in NiFi using the following approach:

Extract the Parent Values: Use a processor like EvaluateJsonPath or UpdateAttribute to extract the parent last two columns' details from the FlowFile attributes and store them in new attributes.

Merge Parent Values with Child Rows: Use a processor like UpdateRecord or ExecuteScript to merge the parent values with the child rows. You'll need to write a custom script or configuration to append the parent values to each child row based on the parent's ID.

Here's a basic example using UpdateRecord processor:

Configure a schema that includes all columns from both parent and child rows.
Use a Record Reader to read the data.
Use a Record Writer to write the data.
In the UpdateRecord processor, use a RecordPath to conditionally update child rows with parent values based on their ID.
Use RecordPath expressions to refer to parent values.

for line in flowfile:
if line.startswith("child"):
# Get parent values from attributes
parent_values = flowfile.getAttribute("parent_values")
# Append parent values to the child row
child_row = line + "," + parent_values
# Write the modified child row to the output FlowFile
output_flowfile.write(child_row)
else:
# If it's a parent row, update the parent values attribute
parent_values = extract_parent_values(line)
flowfile.setAttribute("parent_values", parent_values)

I hope my suggestion is helpful to you.

Best Regard,
angela683h

Re: How to add the parent row values to child row in the NIFI flow file?

kum — Thu, 06 Jun 2024 09:50:48 GMT

Thanks @angela683h i used the same flow to achieve it and thanks for your support

question Re: How to add the parent row values to child row in the NIFI flow file? in Support Questions

How to add the parent row values to child row in the NIFI flow file?

Re: How to add the parent row values to child row in the NIFI flow file?

Re: How to add the parent row values to child row in the NIFI flow file?