Atlas hive is not showing lineage data

[#1]

Hi,

I have created tables for lineage from https://community.hortonworks.com/content/kbentry/36121/using-apache-atlas-to-view-data-lineage.html

I am able to see tables on apache atlas gui..

But I am not able to see lineage data..

lineage.png

[#2]

Hello All,

Right now,I have installed Apache atlas (0.6 version) on HDP-2.4.0.My first question is will Apache atlas work fine on HDP-2.4.0?.As per my understanding Atlas 0.6 version cant be installed on HDP-2.4.0 but somehow,I have installed it on my local machine.

Currently,I am able to see the metadata about hive tables but for that, I have to run import-hive.sh repeatedly which is not expected behavior in Apache Atlas.How can we avoid running import-hive.sh repeatedly?.

and also not getting lineage for hive tables in Atlas?.

I am sure that I have configured atlas correctly for getting metadata from hive,seeing lineage and almost tried all the solutions from we sites but not able to solve the problem.

The links which I have tried for configuration:

1) http://atlas.incubator.apache.org/Configuration.html

2) https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/configu...

please help me,

[#3]

@Manoj Dhake

Download and use Sandbox 2.5, it was just released and is running Atlas 0.7.

https://hortonworks.com/downloads/#tech-preview

This version of Atlas is much better than 0.5 and 0.6 and should come pre-installed. Also, remember that in order to see lineage for an entity, you need to run a process that creates a new entity or modifies the existing entity. The simplest example is running a create table ... select ... from... SQL statement.

[#4]

Hello Vadim,

Thank for identifying thew newer release of Atlas 0.7

I can see that you folks have been very busy creating new features - especially in the area of a Taxonomy. Great Job !!!

So I have written java code which will be highlighting our (IBM's) ability to pull metadata from various sources, and this newer release seems primed for working with your release.

A few observations :

1) While the Apache Atlas interface recognizes newly added entities - the lineage display seems broken or disabled. It only shows a flat line. Any ideas why?

2) Regardless of the Lineage display - my code to call the API with say:

http://192.168.1.133:21000/api/atlas/lineage/hive/table/default.anewsample@Sandbox/inputs/graph

returns me the expected JSON structure of edges/ and verticies

and from each item in the edge i can further query details of the entity using :

http://192.168.1.133:21000/api/atlas/entities/>

Question: How do I now what is the correct order of the lineage? Bear in mind, that JSON returned results can never assure a given order. Any help to understanding the order problem?

Regards,

Russ : rga@us.ibm.com

[#5]

@Russell AndersonSorry it took a while to respond, I have not been back to this question ina while. Could you ask the questions in a separate thread so that the answer is easier to find for others? I will do my best to answer.

[#6]

Hello @Vadim

I tried creaing a new entity via create table as select... and now I can see the lineage, but for an existing entity, what do I have to do to see the lineage?

Thanks

[#7 · Accepted solution]

@Washington Nascimento

Check out the section Importing Hive Metadata from the link: http://atlas.incubator.apache.org/Bridge-Hive.html. You will want to utilize the import-hive.sh script to perform the initial load of metadata.

[#8]

Great @mjohnson!

Thank you !