Support Questions

Find answers, ask questions, and share your expertise

Hdfs to confluent kafka

avatar
Contributor

Hello community,

I want to get the from hdfs to confluent cloud kafka is there any suggestion how to do that.

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hello @Israr ,

Cloudera provides the easiest ways and options to configure such setups through Cloudera Manager (CM) and Streams Messaging Manager (SMM) [1]

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/monitoring-kafka-connect/topics/smm-creating-...

You can configure this pipeline with HDFS or Stateless NiFi Source and Sink connectors [2] & [3]. 

[2]  Public Cloud: https://docs.cloudera.com/runtime/7.2.18/kafka-connect/topics/kafka-connect-connector-nifi-stateless...

[3] PvC: https://docs.cloudera.com/cdp-private-cloud-base/latest/kafka-connect/topics/kafka-connect-connector...

HDFS: https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/kafka-connect/kafka-connect.pdf

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

View solution in original post

7 REPLIES 7

avatar
Community Manager

@Israr Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Kafka experts @ChethanYM @paras  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Contributor

anyone can help ?

avatar
Community Manager

Hi! @Babasaheb @rki_ Do you have any insights here? Thanks!


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Super Collaborator

Hello @Israr ,

Cloudera provides the easiest ways and options to configure such setups through Cloudera Manager (CM) and Streams Messaging Manager (SMM) [1]

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/monitoring-kafka-connect/topics/smm-creating-...

You can configure this pipeline with HDFS or Stateless NiFi Source and Sink connectors [2] & [3]. 

[2]  Public Cloud: https://docs.cloudera.com/runtime/7.2.18/kafka-connect/topics/kafka-connect-connector-nifi-stateless...

[3] PvC: https://docs.cloudera.com/cdp-private-cloud-base/latest/kafka-connect/topics/kafka-connect-connector...

HDFS: https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/kafka-connect/kafka-connect.pdf

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

avatar
Contributor

Israr_0-1727198765742.png

i have install smm on my cdp priavate cluster connector option is not showing , is any other service also need i am using cdp 7.4.4 or i have chnage cdp version

avatar
Super Collaborator

Hello @Israr ,

You should verify that both Kafka and Kafka Connect are running and in a healthy state.

Thank you. 

 

avatar
Contributor

Yes, It is running i am able to create the topic