Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Atlas hive is not showing lineage data

avatar
Super Collaborator

Hi,

I have created tables for lineage from https://community.hortonworks.com/content/kbentry/36121/using-apache-atlas-to-view-data-lineage.html

I am able to see tables on apache atlas gui..

But I am not able to see lineage data..

lineage.png

1 ACCEPTED SOLUTION

avatar
Rising Star
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
7 REPLIES 7

avatar
Super Collaborator

Hello All,

Right now,I have installed Apache atlas (0.6 version) on HDP-2.4.0.My first question is will Apache atlas work fine on HDP-2.4.0?.As per my understanding Atlas 0.6 version cant be installed on HDP-2.4.0 but somehow,I have installed it on my local machine.

Currently,I am able to see the metadata about hive tables but for that, I have to run import-hive.sh repeatedly which is not expected behavior in Apache Atlas.How can we avoid running import-hive.sh repeatedly?.

and also not getting lineage for hive tables in Atlas?.

I am sure that I have configured atlas correctly for getting metadata from hive,seeing lineage and almost tried all the solutions from we sites but not able to solve the problem.

The links which I have tried for configuration:

1) http://atlas.incubator.apache.org/Configuration.html

2) https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/configu...

please help me,

avatar
Guru

@Manoj Dhake

Download and use Sandbox 2.5, it was just released and is running Atlas 0.7.

https://hortonworks.com/downloads/#tech-preview

This version of Atlas is much better than 0.5 and 0.6 and should come pre-installed. Also, remember that in order to see lineage for an entity, you need to run a process that creates a new entity or modifies the existing entity. The simplest example is running a create table ... select ... from... SQL statement.

avatar
Explorer

Hello Vadim,

Thank for identifying thew newer release of Atlas 0.7

I can see that you folks have been very busy creating new features - especially in the area of a Taxonomy. Great Job !!!

So I have written java code which will be highlighting our (IBM's) ability to pull metadata from various sources, and this newer release seems primed for working with your release.

A few observations :

1) While the Apache Atlas interface recognizes newly added entities - the lineage display seems broken or disabled. It only shows a flat line. Any ideas why?

2) Regardless of the Lineage display - my code to call the API with say:

http://192.168.1.133:21000/api/atlas/lineage/hive/table/default.anewsample@Sandbox/inputs/graph

returns me the expected JSON structure of edges/ and verticies

and from each item in the edge i can further query details of the entity using :

http://192.168.1.133:21000/api/atlas/entities/>

Question: How do I now what is the correct order of the lineage? Bear in mind, that JSON returned results can never assure a given order. Any help to understanding the order problem?

Regards,

Russ : rga@us.ibm.com

avatar
Guru
@Russell AndersonSorry it took a while to respond, I have not been back to this question ina while. Could you ask the questions in a separate thread so that the answer is easier to find for others? I will do my best to answer.

avatar
Contributor

Hello @Vadim

I tried creaing a new entity via create table as select... and now I can see the lineage, but for an existing entity, what do I have to do to see the lineage?

Thanks

avatar
Rising Star
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Contributor

Great @mjohnson!

Thank you !