Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas lineage REST API and UI query on depth

Highlighted

Atlas lineage REST API and UI query on depth

Contributor

Please could someone help me describe/define for me what exactly is meant by 'depth' in lineage. I understand the default is 3. Is this 3 <hive>_processes in each direction?

curl -k -u admin:admin -H "Content-type:application/json" -X GET https://atlas.url:atlas_port/api/atlas/v2/lineage/a2303e0a-e3ff-4823-8c45-c59a86438a77?depth=7 | python -m json.tool

Also, how would I write a curl query that would define both the depth and the direction? Currently I can only seem to be able to do one other other. Below is the direction defined.

curl -k -u admin:admin -H "Content-type:application/json" -X GET https://atlas.url:atlas_port/api/atlas/v2/lineage/a2303e0a-e3ff-4823-8c45-c59a86438a77?direction=INP... | python -m json.tool

On the UI side, is the lineage graph defaulted to a depth of 3? Is there any way I can change the output to a custom depth?

Is there a way of filtering the lineage graph, for example by tag? I have many, many temp staging tables that are showing in the lineage graph in our staging area, and really making the lineage graph a mess. I'm not joking when I a say there are some tables with a 'line' an inch think because of the number of temp staging tables!!!

2 REPLIES 2

Re: Atlas lineage REST API and UI query on depth

Expert Contributor

@Laura Ngo, lineage 'depth' is the number of hops on DataSet in each direction. The default lineage depth is 3 and default lineage direction is BOTH.

To specify both lineage direction and depth you could use:

curl -k -u admin:admin -H "Content-type:application/json" -X GET http://localhost:21000/api/atlas/v2/lineage/df8f62df-d956-4c82-828b-5e8ebc160d13?depth=5&direction=I... | python -m json.tool

On the UI side, we don't pass the depth parameter as argument to the REST, hence it defaults to depth=3 in server-side. Currently we don't have a way to filter lineage entities based on tags or any filters, but this is a good use case and we will try to add this in future.

Re: Atlas lineage REST API and UI query on depth

@Laura Ngo

I've found a workaround to show more than 3 hops on UI, I desperately needed that:

I changed this file:
/usr/hdp/2.6.0.3-8/atlas/server/webapp/atlas/js/collection/VLineageList.js

and added depth parameter that @Sarath Subramanian suggested:

...

getLineage: function(id, options) {

var url = UrlLinks.lineageApiUrl(id) + '?depth=99';

...

PS: although changing source code is not recommended, I don't see any risk with such a simple change and benefits are great!!!