Support Questions

Find answers, ask questions, and share your expertise

Hdfs to confluent kafka

avatar
Contributor

Hello community,

I want to get the from hdfs to confluent cloud kafka is there any suggestion how to do that.

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hello @Israr ,

Cloudera provides the easiest ways and options to configure such setups through Cloudera Manager (CM) and Streams Messaging Manager (SMM) [1]

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/monitoring-kafka-connect/topics/smm-creating-...

You can configure this pipeline with HDFS or Stateless NiFi Source and Sink connectors [2] & [3]. 

[2]  Public Cloud: https://docs.cloudera.com/runtime/7.2.18/kafka-connect/topics/kafka-connect-connector-nifi-stateless...

[3] PvC: https://docs.cloudera.com/cdp-private-cloud-base/latest/kafka-connect/topics/kafka-connect-connector...

HDFS: https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/kafka-connect/kafka-connect.pdf

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

View solution in original post

7 REPLIES 7

avatar
Community Manager

@Israr Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Kafka experts @ChethanYM @paras  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Contributor

anyone can help ?

avatar
Community Manager

Hi! @Babasaheb @rki_ Do you have any insights here? Thanks!


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Super Collaborator

Hello @Israr ,

Cloudera provides the easiest ways and options to configure such setups through Cloudera Manager (CM) and Streams Messaging Manager (SMM) [1]

[1] https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/monitoring-kafka-connect/topics/smm-creating-...

You can configure this pipeline with HDFS or Stateless NiFi Source and Sink connectors [2] & [3]. 

[2]  Public Cloud: https://docs.cloudera.com/runtime/7.2.18/kafka-connect/topics/kafka-connect-connector-nifi-stateless...

[3] PvC: https://docs.cloudera.com/cdp-private-cloud-base/latest/kafka-connect/topics/kafka-connect-connector...

HDFS: https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/kafka-connect/kafka-connect.pdf

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

avatar
Contributor

Israr_0-1727198765742.png

i have install smm on my cdp priavate cluster connector option is not showing , is any other service also need i am using cdp 7.4.4 or i have chnage cdp version

avatar
Super Collaborator

Hello @Israr ,

You should verify that both Kafka and Kafka Connect are running and in a healthy state.

Thank you. 

 

avatar
Contributor

Yes, It is running i am able to create the topic