Support Questions
Find answers, ask questions, and share your expertise

Atlas lineage REST API and UI query on depth

Contributor

Please could someone help me describe/define for me what exactly is meant by 'depth' in lineage. I understand the default is 3. Is this 3 <hive>_processes in each direction?

curl -k -u admin:admin -H "Content-type:application/json" -X GET https://atlas.url:atlas_port/api/atlas/v2/lineage/a2303e0a-e3ff-4823-8c45-c59a86438a77?depth=7 | python -m json.tool

Also, how would I write a curl query that would define both the depth and the direction? Currently I can only seem to be able to do one other other. Below is the direction defined.

curl -k -u admin:admin -H "Content-type:application/json" -X GET https://atlas.url:atlas_port/api/atlas/v2/lineage/a2303e0a-e3ff-4823-8c45-c59a86438a77?direction=INP... | python -m json.tool

On the UI side, is the lineage graph defaulted to a depth of 3? Is there any way I can change the output to a custom depth?

Is there a way of filtering the lineage graph, for example by tag? I have many, many temp staging tables that are showing in the lineage graph in our staging area, and really making the lineage graph a mess. I'm not joking when I a say there are some tables with a 'line' an inch think because of the number of temp staging tables!!!

2 REPLIES 2

Re: Atlas lineage REST API and UI query on depth

Expert Contributor

@Laura Ngo, lineage 'depth' is the number of hops on DataSet in each direction. The default lineage depth is 3 and default lineage direction is BOTH.

To specify both lineage direction and depth you could use:

curl -k -u admin:admin -H "Content-type:application/json" -X GET http://localhost:21000/api/atlas/v2/lineage/df8f62df-d956-4c82-828b-5e8ebc160d13?depth=5&direction=I... | python -m json.tool

On the UI side, we don't pass the depth parameter as argument to the REST, hence it defaults to depth=3 in server-side. Currently we don't have a way to filter lineage entities based on tags or any filters, but this is a good use case and we will try to add this in future.

Re: Atlas lineage REST API and UI query on depth

@Laura Ngo

I've found a workaround to show more than 3 hops on UI, I desperately needed that:

I changed this file:
/usr/hdp/2.6.0.3-8/atlas/server/webapp/atlas/js/collection/VLineageList.js

and added depth parameter that @Sarath Subramanian suggested:

...

getLineage: function(id, options) {

var url = UrlLinks.lineageApiUrl(id) + '?depth=99';

...

PS: although changing source code is not recommended, I don't see any risk with such a simple change and benefits are great!!!