Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there a way to automatically add missing hive tables to apache atlas ?

Is there a way to automatically add missing hive tables to apache atlas ?

New Contributor

Hi,

I am using atlas for lineage and data governance and all integrations with hive,spark and sqoop are up and running. But I want to know if there is a way to automate the process of adding all the missing tables to atlas at the end of every day in order to make sure that no data is missing.

 

I wrote a script that gets the list of all tables from hive metastore and got the list of missing table from atlas by hitting its api.

The plan was to call "import-hive.sh"(Inbuilt hive bridge of atlas) over the missing tables but the script gets stuck at an intermediate phase asking for input of username and password.so i am unable to automate it.

 

I even tried things like these and nothing worked.

printf "admin\nadmin" | sh import-hive.sh --database dummydb --table final

 

can someone help me with this?

1 REPLY 1
Highlighted

Re: Is there a way to automatically add missing hive tables to apache atlas ?

Cloudera Employee

My first question would be are your Hive tables missing from both Basic and DSL search?

 

Also, Atlas should automatically fetch tables created in Hive, if that isn't happening you should find out why. This article can help you troubleshoot:

https://community.cloudera.com/t5/Community-Articles/Atlas-Install-Validation-and-troubleshooting-ti...

Don't have an account?
Coming from Hortonworks? Activate your account here