question Re: Hdfs to confluent kafka in Support Questions

Hdfs to confluent kafka

Israr — Wed, 18 Sep 2024 20:49:05 GMT

Hello community,

I want to get the from hdfs to confluent cloud kafka is there any suggestion how to do that.

Re: Hdfs to confluent kafka

DianaTorres — Wed, 18 Sep 2024 21:26:17 GMT

@Israr Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Kafka experts @ChethanYM @paras who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: Hdfs to confluent kafka

Israr — Sat, 21 Sep 2024 18:28:44 GMT

anyone can help ?

Re: Hdfs to confluent kafka

DianaTorres — Mon, 23 Sep 2024 23:08:46 GMT

Hi! @Babasaheb @rki_ Do you have any insights here? Thanks!

Re: Hdfs to confluent kafka

Babasaheb — Tue, 24 Sep 2024 06:35:28 GMT

Hello @Israr ,

Cloudera provides the easiest ways and options to configure such setups through Cloudera Manager (CM) and Streams Messaging Manager (SMM) [1]

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/monitoring-kafka-connect/topics/smm-creating-a-connector.html

You can configure this pipeline with HDFS or Stateless NiFi Source and Sink connectors [2] & [3].

[2] Public Cloud: https://docs.cloudera.com/runtime/7.2.18/kafka-connect/topics/kafka-connect-connector-nifi-stateless.html

[3] PvC: https://docs.cloudera.com/cdp-private-cloud-base/latest/kafka-connect/topics/kafka-connect-connector-nifi-stateless.html

HDFS: https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/kafka-connect/kafka-connect.pdf

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

Re: Hdfs to confluent kafka

Israr — Tue, 24 Sep 2024 17:27:57 GMT

i have install smm on my cdp priavate cluster connector option is not showing , is any other service also need i am using cdp 7.4.4 or i have chnage cdp version

Re: Hdfs to confluent kafka

Babasaheb — Wed, 25 Sep 2024 13:02:28 GMT

Hello @Israr ,

You should verify that both Kafka and Kafka Connect are running and in a healthy state.

Thank you.

Re: Hdfs to confluent kafka

Israr — Wed, 25 Sep 2024 13:05:25 GMT

Yes, It is running i am able to create the topic