Processor work JoinEnrichment

[#1]

I created the following project:

and I want to connect 2 files by field "id"

Processor name "GenerateRecord" has the following settings:

Processor name "GenerateRecord2" has the following settings:

Processor name "JoinEnrichment" has the following settings:

The error occurs when "JoinEnrichment" works, I can’t understand why there, I tried everything I could

[#2]

Hi @skoch244 ,

You are getting this error because you are suppose to use JoinEnrichment in conjunction with ForkEnrichment processor as the documentation of those processors specify:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apach...

The error details indicate that it cant find 'original' and 'enrichment' flowfiles. Those terms are setup and created by the FormEnrichment processor through its relationships:

enrichment & original.

Im not sure what your actual flow looks like and what are the actual source for each of the data sources but just using GenerateRecord for both dataset wont work because this processor cant take upstream relationship which doesnt apply to the pattern of using the Fork\JoinEnrichment processors. The link above will have an example that will help you understand how those processors work. If you need more help with your actual flow please provide more information on the actual scenario you have.

If you find this helpful please accept solution.

Thanks

[#3]

Hi @SAMSAL ,
Thanks for the link you provided in your answer, it helped me understand the use of forkEnrichement, while I was trying to figure out the use of joinEnrichement. In my case I am first reading a column from a database table, then I am reading a column from a csv file, I have to join the db column with the csv column in a certain way to get my desired result.

So, what is done here is, the data from db is provided to fork enrichment, it makes two copies of it, one is called 'original' and the other is called 'enrichment'. Though I am not performing anything on the enrichment flow file, rather I am overwriting it with my csv data, because that's what I need to do later in join enrichment i.e. to join the data from db to csv data.

I have achieved the same using another way previously,  by adding a property to each set of data i.e. db one and csv one to identify them separately while joining them, then I am merging them and then using queryRecord to finally join them and extract the result set I need.

However, if you have suggestions to achieve this in a better way, do share with us, thanks.

[#4]

Hi @SS_Jin ,

Glad to hear that my post helped. Its really hard to suggest something specially when I dont have all the details of what you are trying to do but from what I read, I think Join\Fork Enrichment would work better in these scenarios. The mergerecord way could be problematic when you are reading multiple sources and multiple CSVs where merge behavior can be unpredictable. Also depending on what type of enrichment you are trying to do and how complex its but if you have one to one mapping between record in the DB vs CSV and you are trying to override some data or add new one then you might also consider the LookupRecord processor to simplify your data flow where you dont have to use branching to read and then merge the different sources which might endup saving you some overhead.

https://community.cloudera.com/t5/Community-Articles/Data-flow-enrichment-with-NiFi-part-1-LookupRec...