Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Lineage data failed to fetch in Atlas Web UI for large tables

Solved Go to solution

Lineage data failed to fetch in Atlas Web UI for large tables

On my atlas web UI, selecting a table takes too much time to fetch lineage and incase of big tables after loading for too long it just displays "No Lineage Data Found" on checking the requests made which are taking long I found one REST API's being called 2 times which take somewhere between 3-5 mins. Sometimes it successfully fetches data and displays lineage but in case of too big tables no lineage data appears.

The API which is being called twice is:-

Request URL:http://hostname:port/api/atlas/lineage/c797bffa-37d0-4b23-b168-998ffc4855e3/inputs/graph?_=1487780877854

JSON Response

{"requestId":"qtp1177377518-69414 - 3e3301e8-4944-429e-96fe-9ef640684372","results":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"__tempQueryResultStruct523","values":{"vertices":{},"edges":{}}}} 

Time Taken 3.7 minutes.

I guess the request is taking too long to process and hence empty vertices and edges are recieved in the JSON Response. Is there someway I can increase the time for this request to be processed or better improve the performance altogether by using some caching mechanism or something.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

Expert Contributor

@Poorvi Sachar

Same API is not called twice . It must be

http://hostname:port/api/atlas/lineage/c797bffa-37d0-4b23-b168-998ffc4855e3/inputs/graph

and

http://hostname:port/api/atlas/lineage/c797bffa-37d0-4b23-b168-998ffc4855e3/outputs/graph

Can you please check ? If it's not , it is a bug possibly.

View solution in original post

9 REPLIES 9
Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

@Poorvi Sachar

What does the atlas application log say, when this request is made? which version of HDP is this? Is it possible to attach the application log here for debugging?

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

As a am a newbie could you please tell me which logs to attach here..

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

Atlas logs are stored in application.log file which can be found at <ATLAS_HOME>/logs/application.log.

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

@Poorvi Sachar Are you still seeing this issue? if yes, please attach the application logs?

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

@Ayub Khan

Yes for few tables the lineage result shows as - "Lineage data not found". I am not sure that whether it has nothing to show or it just fetches null edges and vertices because the api call takes a longer duration to complete.

Do you need to see some specific section of the application logs? Also I did not find anything related to the issue but incase you still need it then maybe you can tell me what exactly are you looking for as the logs contain some extra information.

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

Expert Contributor

@Poorvi Sachar

Same API is not called twice . It must be

http://hostname:port/api/atlas/lineage/c797bffa-37d0-4b23-b168-998ffc4855e3/inputs/graph

and

http://hostname:port/api/atlas/lineage/c797bffa-37d0-4b23-b168-998ffc4855e3/outputs/graph

Can you please check ? If it's not , it is a bug possibly.

View solution in original post

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

yeah sorry its two different apis...could still you please help me figure out why lineage data is not being fetched or how could i really check in the db it there is no data....also is there some method to improve this performance...as it usually takes 3-4 mins to fetch data for small tables which is way too much

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

Expert Contributor

How long does it take for other APIs ? Atlas server might be residing in a remote location and you might be pinging from your local machine hence there can be network delay. Can you please login to the machine and do curl GET of the same APIs and add -w to curl to check the time taken ?

Also,

>> I guess the request is taking too long to process and hence empty vertices and edges are recieved in the JSON Response.

This is not true. UI says "Lineage not found" when there is really no lineage data. Truthfulness of this API response doesn't depend on the time taken. We should debug why the API call takes long time.

Highlighted

Re: Lineage data failed to fetch in Atlas Web UI for large tables

Expert Contributor

@Poorvi Sachar

If it shows "Lineage not found" , it is not an exception or error . It means that actually there is no Lineage.

Don't have an account?
Coming from Hortonworks? Activate your account here