Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to avoid AttributesTOJSON and ConvertJSONToSQL being repeated

Highlighted

How to avoid AttributesTOJSON and ConvertJSONToSQL being repeated

Explorer

XML->JSON->oracle tables.

 

My flow Get the JSON file and form a required JSON format(JoltTransform) for 1 table and route to operations  - ADD/MODIFY/REMOVE based on JSON file content

for db insert - from routeAttributes ->AttributeToJSON->convertJSONtoSQL ->putSQL

ADD- Insert,MODIFY-Update and REMOVE-Delete from table.

So here for one table - I have three AttributeToJSON and convertJSONtoSQL

 

2020-03-31 14_54_51-NiFi Flow.png

 

2020-03-31 14_51_19-NiFi Flow.png

2 REPLIES 2
Highlighted

Re: How to avoid AttributesTOJSON and ConvertJSONToSQL being repeated

@MPraveen   Very good question here.

 

First, it is okay to have duplicates of processors, especially when the data flow branches are different or far away from each other.   Sometimes these duplicates are required as they might have slight differences in the downstream path.

 

That said,  if you can structure your flow to utilize single examples of processors this is easier to understand and follow.  In order to do this the processors have to be configured identically.  

 

 In your example below,  are there customizations to AttributesToJson and ConvertJsonToSql specific to each branch: ADD,MODIFY,REMOVE?      If not try to combine all 3 into a single new test branch (you can route each more than one time).  Route all 3 to this new branch. Test it.  If it works, remove the other 3 branches.

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Re: How to avoid AttributesTOJSON and ConvertJSONToSQL being repeated

Explorer

Thanks for your answer Steven. Yes there are customizations to AttributesToJson and ConvertJsonToSql.  

Don't have an account?
Coming from Hortonworks? Activate your account here