Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi integration with Atlas in kerberos environment

Highlighted

Nifi integration with Atlas in kerberos environment

New Contributor

I am using HDP 3.0.1 and HDF 3.3.0 in kerberos enabled environment. I want to achieve Nifi-Atlas Integration in a kerberos environment. I followed steps specified at https://hortonworks.com/blog/hdf-3-1-blog-series-part-6-introducing-nifi-atlas-integration/ .
Now, when i enable ReportLineageToAtlas reporting task and run a simple flow involving generateFlowFile and putHDFS processors, i am facing "org.apache.kafka.common.KafkaException: Failed to construct kafka producer" caused by "Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.kafka.common.protocol.SecurityProtocol.PLAINTEXTSASL" . The error is shared in screenshot as well.

108831-kafkaerror.png


On exploring the code, i found out that Nifi 1.8 code uses atlas-client 0.8.1 which in turn uses kafka 0.10.0.0 client. But the HDP 3.0.1 environment comes with Kafka 2.0. According to me, the error is because of this version mis-match.

Kindly help me out in resolving the issue.

I see 2 ways :

1. Either building nifi 1.8 with atlas client 1.0.0 but that involves lots of changes on nifi 1.8 code.

2. Either building atlas-client 0.8.1 with kafka-client 1.0.0

But before starting on same i would like to know the community view whether i am guessing the reason correctly. Or this is something Hortonworks has already handled(since i am looking at nifi 1.8 and atlas 0.8.1 code from git not hortonworks repository and hence dont know whether they have made those changes or not).

Kindly help.