question Re: NIFI ORACLE and SQL CDC in Support Questions

NIFI ORACLE and SQL CDC

moahmedhassaan — Wed, 12 Apr 2023 11:43:28 GMT

is there is processor from oracle and oracle cdc now in cloudera ???

Re: NIFI ORACLE and SQL CDC

ckumar — Wed, 12 Apr 2023 13:33:50 GMT

There is No specific processor built only for Oracle but if you are talking about Oracle DB then one can use ExecuteSQL/PutSQL with DBCPConnectionPool Controller where DBCPConnectionPool controller ie generic implementation to connect any DB and it requires a local copy of Database specific client driver and driver class name.

Please refer https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-dbcp-service-nar/1.21.0/org.apache.nifi.dbcp.DBCPConnectionPool/index.html

If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Chandan

Re: NIFI ORACLE and SQL CDC

cotopaul — Wed, 12 Apr 2023 13:51:31 GMT

hi @moahmedhassaan,

Regarding a CDC in NiFi for an Oracle instance, I highly recommend the following article as it guides you step by step in order to perform a partial CDC, only for UPDATE and INSERT:
https://murtazak.medium.com/mimic-an-upsert-in-oracle-using-nifi-bb112dc1d6ab

This solution will not work for DELETE, but based on that example you can create your own. This will however eat lots of resources if not configured properly.

Re: NIFI ORACLE and SQL CDC

moahmedhassaan — Sun, 06 Aug 2023 16:01:38 GMT

thanks for your reply but my question is how to implement CDC (incremental load ) without having an incremental column .

Re: NIFI ORACLE and SQL CDC

TimothySpann — Mon, 07 Aug 2023 01:07:23 GMT

You need to look at CDC tools like Cloudera Kafka Connect with Debezium or Cloudera SQL Streaming Builder with Debezium.

Kafka Connect

https://docs.cloudera.com/runtime/7.2.17/kafka-connect/topics/kafka-connect-connector-debezium-oracle.html

SSB

https://docs.cloudera.com/csa/1.10.0/how-to-ssb/topics/csa-ssb-cdc-connectors.html

Or you can do Oracle GoldenGate with Kafka.

I have some examples in this article

https://medium.com/cloudera-inc/cdc-not-cat-data-capture-e43713879c03