Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

when i configure hive hook and create table in hive that didn't add in atlas automatically and not appear in atlas hive-tables entity

when i configure hive hook and create table in hive that didn't add in atlas automatically and not appear in atlas hive-tables entity

New Contributor

i configured hive hook for atlas in hive-site.xml by the following:

atlas.hook.hive.synchronous = true

hive.exec.post.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook

atlas.cluster.name = dpf


i use ambari for configuration

two kafka topic is created:

ATLAS_ENTITIES

ATLAS_HOOK


and when i created the table in hive for example:

create table br(full_name string, ssn string, location string);


and then i run this command:

/usr/hdp/2.6.5.0-292/kafka/bin/kafka-console-consumer.sh --zookeeper dlm01.sic:2181,dlm02.sic:2181,dlm03.sic:2181 --topic ATLAS_HOOK --from-beginning

i see the json format of text for create table br. this means hive send data to ATLAS_HOOK topic and it works correctly.

but when i run this command:

/usr/hdp/2.6.5.0-292/kafka/bin/kafka-console-consumer.sh --zookeeper dlm01.sic:2181,dlm02.sic:2181,dlm03.sic:2181 --topic ATLAS_ENTITIES --from-beginning

the didn't see anything about br table.

why table not appear in atlas hive-tables ? what is my wrong?


notice:

when i manually import hive metadata in atlas with import-hive.sh i see the information about br table in two kafka topics(ATLAS_ENTITIES, ATLAS_HOOK ) and br table appear in atlas. but automatically not work

2 REPLIES 2

Re: when i configure hive hook and create table in hive that didn't add in atlas automatically and not appear in atlas hive-tables entity

Expert Contributor

@Ahmad Hajitorab, There might be some issue in Atlas Notification kafka consumer. please check atlas-application.log to find the root cause of issue.

Re: when i configure hive hook and create table in hive that didn't add in atlas automatically and not appear in atlas hive-tables entity

New Contributor

@Ahmad Hajitorab ,

Try to run sample metadata in Atlas.

su atlas -c '/usr/hdp/current/atlas-server/bin/quick_start.py' after then check application.log. Questions:- Have you enabled Plugin's and Have you set Audit to Solr? You can also run import-hive.sh. Please share your logs details.