Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas hive is not showing lineage data

Solved Go to solution
Highlighted

Atlas hive is not showing lineage data

Expert Contributor

Hi,

I have created tables for lineage from https://community.hortonworks.com/content/kbentry/36121/using-apache-atlas-to-view-data-lineage.html

I am able to see tables on apache atlas gui..

But I am not able to see lineage data..

lineage.png

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Atlas hive is not showing lineage data

Explorer

@Washington Nascimento

Check out the section Importing Hive Metadata from the link: http://atlas.incubator.apache.org/Bridge-Hive.html. You will want to utilize the import-hive.sh script to perform the initial load of metadata.

View solution in original post

7 REPLIES 7
Highlighted

Re: Atlas hive is not showing lineage data

Expert Contributor

Hello All,

Right now,I have installed Apache atlas (0.6 version) on HDP-2.4.0.My first question is will Apache atlas work fine on HDP-2.4.0?.As per my understanding Atlas 0.6 version cant be installed on HDP-2.4.0 but somehow,I have installed it on my local machine.

Currently,I am able to see the metadata about hive tables but for that, I have to run import-hive.sh repeatedly which is not expected behavior in Apache Atlas.How can we avoid running import-hive.sh repeatedly?.

and also not getting lineage for hive tables in Atlas?.

I am sure that I have configured atlas correctly for getting metadata from hive,seeing lineage and almost tried all the solutions from we sites but not able to solve the problem.

The links which I have tried for configuration:

1) http://atlas.incubator.apache.org/Configuration.html

2) https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/configu...

please help me,

Re: Atlas hive is not showing lineage data

Guru

@Manoj Dhake

Download and use Sandbox 2.5, it was just released and is running Atlas 0.7.

https://hortonworks.com/downloads/#tech-preview

This version of Atlas is much better than 0.5 and 0.6 and should come pre-installed. Also, remember that in order to see lineage for an entity, you need to run a process that creates a new entity or modifies the existing entity. The simplest example is running a create table ... select ... from... SQL statement.

Highlighted

Re: Atlas hive is not showing lineage data

Explorer

Hello Vadim,

Thank for identifying thew newer release of Atlas 0.7

I can see that you folks have been very busy creating new features - especially in the area of a Taxonomy. Great Job !!!

So I have written java code which will be highlighting our (IBM's) ability to pull metadata from various sources, and this newer release seems primed for working with your release.

A few observations :

1) While the Apache Atlas interface recognizes newly added entities - the lineage display seems broken or disabled. It only shows a flat line. Any ideas why?

2) Regardless of the Lineage display - my code to call the API with say:

http://192.168.1.133:21000/api/atlas/lineage/hive/table/default.anewsample@Sandbox/inputs/graph

returns me the expected JSON structure of edges/ and verticies

and from each item in the edge i can further query details of the entity using :

http://192.168.1.133:21000/api/atlas/entities/>

Question: How do I now what is the correct order of the lineage? Bear in mind, that JSON returned results can never assure a given order. Any help to understanding the order problem?

Regards,

Russ : rga@us.ibm.com

Highlighted

Re: Atlas hive is not showing lineage data

Guru
@Russell AndersonSorry it took a while to respond, I have not been back to this question ina while. Could you ask the questions in a separate thread so that the answer is easier to find for others? I will do my best to answer.
Highlighted

Re: Atlas hive is not showing lineage data

Explorer

Hello @Vadim

I tried creaing a new entity via create table as select... and now I can see the lineage, but for an existing entity, what do I have to do to see the lineage?

Thanks

Highlighted

Re: Atlas hive is not showing lineage data

Explorer

@Washington Nascimento

Check out the section Importing Hive Metadata from the link: http://atlas.incubator.apache.org/Bridge-Hive.html. You will want to utilize the import-hive.sh script to perform the initial load of metadata.

View solution in original post

Highlighted

Re: Atlas hive is not showing lineage data

Explorer

Great @mjohnson!

Thank you !

Don't have an account?
Coming from Hortonworks? Activate your account here