Atlas Tag not showing up in Ranger, getting warning in tagsync.log. How to fix?

[#1]

Hi,

Tags that I create in Atlas, are not showing up in Ranger. Tagsync service is running, however I notice the following Warning in the tagsync log:

28 Dec 2016 14:13:47  WARN ClientUtils$ [ranger_entities_consumer_wbekkerdv2-1482932174435-afd1c862-leader-finder-thread] - 89 Fetching topic metadata with correlation id 10939 for topics [Set(ATLAS_ENTITIES)] from broker [BrokerEndPoint(1001,wbekkerdv0.field.hortonworks.com,6667)] failed
java.nio.channels.ClosedChannelException
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:122)
        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:82)
        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$doSend(SyncProducer.scala:81)
        at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
        at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)

Any pointers?

[#2 · Accepted solution]

@Ward Bekker From the log, it looks like the tagsync thread listening on Kafka events has failed. You may want to use kafka console consumer to see if the Atlas events are coming through by running 'kafka-console-consumer.sh --zookeeper <zookeeper-host-ip>:2181 --topic ATLAS_ENTITIES --from-beginning'. If the Atlas events are shown up, then please check tagsync configuration, especially, atlas-application.properties file, and try restarting tagsync.

[#3]

`atlas-application.properties` referred to a `rangertagsync.service.keytab` that wasn't generated for that host. Regenerating the Kerberos keytabs from Ambari fixed it. Thx!

[#4]

make sure you have the kafka policy create in Ranger, it is not created automatically as part of Atlas installation and need to be done manually.

ATLAS_HOOK topic:

User: atlas, Privileges: Consume, Create

Group: public, Privileges: Publish, Create

ATLAS_ENTITIES topic:

User: atlas, Privileges: Publish, Create

Group: public, Privileges: Consume, Create

Also check HBase policy and make sure you see 2 policies for atlas-titan and ATLAS_ENTITY_AUDIT_EVENTS table (these should get create automatically).

Once the policies are in place, hbase, kafka, ranger and atlas. You should have it in Ranger.