Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka Connect HDFS connector (HDInsights)

Kafka Connect HDFS connector (HDInsights)

New Contributor

Hi all,

 

We are getting the following issue with the HDFS Connect Kafka Connector

 

[2019-05-08 10:10:41,820] INFO Opening record writer for: hdfs://headnodehost//topics//+tmp/rstest/partition=1/b9f2fd9d-9dcd-4853-b5a1-dabcf9928884_tmp.avro (io.confluent.connect.hdfs.avro.AvroRecordWriterProvider:66)

and no data is written, when we close the connector, this error occurs

[2019-05-08 10:14:51,370] ERROR Error discarding temp file hdfs://headnodehost//topics//+tmp/rstest/partition=1/b9f2fd9d-9dcd-4853-b5a1-dabcf9928884_tmp.avro for rstest-1 partition=1 when closing TopicPartitionWriter: (io.confluent.connect.hdfs.TopicPartitionWriter:451)
org.apache.kafka.connect.errors.DataException: java.nio.channels.ClosedChannelException
        at 

Please can someone help?

Thanks

 

1 REPLY 1
Highlighted

Re: Kafka Connect HDFS connector (HDInsights)

Rising Star

Hello @RichardJG ,

 

Thanks for posting your query.

 

Based on the documentation, at present Kafka connect is not supported

 

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_known_issues.html#xd_583c10bfdbd326...

 

Is it possible for you to post the complete exception?

Thanks,
Satz
Don't have an account?
Coming from Hortonworks? Activate your account here