Incremetnals Update and Delete Using NiFi

Question

Hi Team,Please help me on the below issueI am ingesting the Data From Oracle to PostgresSql Using NiFi.We have huge data and continuously generating from sources, I want to update attributes and delete attributes using NiFi My Flow is: QueryDatabaseRecord-->UpdateAttribute-->PutDatabaseRecordwhen matched record found on the final table then define which action you need to take either Update (or) Deleteif the record not matched in the final dataset then insert the record.Appreciate your help.Thank you,

Shu_ashu · Accepted Answer

@DeppuIf you are able to identify insert/updates then set statement.type attribute to the flowfile then based on the attribute value PutDatabaseRecord processor will run Update/Insert/Delete Statements.-This is kind of hacky way to do this:If you are not able to identify inserts/updates then split the records into each record individually using SplitRecord processor then run two/three successive PutDatabaseRecord processors. You need to make decision how to identify Deletes/Updates. because if insert fails then you can try with either update statement type (or) delete statement type.Flow:1. FirstPutDatabaseRecord //insert statementtype use failure connection to next PutDatabaseRecord2.SecondPutDatabaseRecord //update statementtype,use failure connection to next PutDatabaseRecord.3.ThirdPutDatabaseRecord //delete statementtypeNote: I'm guessing this flow but rearrange these processors as per your logic.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.