About Debu77

Debu77 · ‎08-20-2019

Hi, Greetings and wish you are doing good ! I am using Hadoop 3.1 version. I am connected Atlas to our Enterprise Kafka Cluster. My objective is to visualise the metadata of the Kafka Topics within Atlas User Interface. With the above objective in mind, I would like to ask the following questions: 1. What kind of data lineage facility for Kafka topics is available in Atlas ? For example, I have some metadata listed in a Kafka topic and can I visualise the same in Atlas 2. Just like Hive tables, can we have a similar data lineage across Kafka Topics? 3. What kind of Data Governance use cases can be catered through Kafka in Atlas ? 4. Does Atlas uses Kafka to communicate with Rangers ? Will appreciate for your feedback as per your earliest convenience please. Thanks, Debu

vsuvagia · ‎07-03-2019

@Debabrata Ghosh, integrating external Kafka to Atlas should be possible, In addition In addition you will need to create and provide appropriate permissions in Kafka for topics ATLAS_HOOKS and ATLAS_ENTITIES which will be used by Atlas and Atlas hook services for respective users. Additionally, kafka configurations will need to be put in place in Atlas and the Atlas hook services in the atlas-application.properties atlas.kafka.bootstrap.servers and atlas.kafka.zookeeper.connect. Mainly, Kafka brokers configuration and Kafka-Zookeeper quorum should be added in the atlas-application.properties to get started with. If Kafka cluster is kerberized then Atlas will need to use the same KDC as Kafka cluster.

naveen_dasharat · ‎05-15-2019

Hi, Did you found any solution for the above issue, as I am also facing same problem. Thanks in advance.

Online	Offline
Last Visited	‎09-27-2019 06:50 AM

Date Registered	‎06-02-2019 02:20 AM
Last Visited	‎09-27-2019 06:50 AM
Total Messages Posted	4

Query regarding Data Lineage for Kafka Topics in A...

Re: Can we integrate an external Kafka cluster wit...

Re: Hortonworks 2.5 - Hive ACID Insert is getting ...