Support Questions
Find answers, ask questions, and share your expertise

Apache Atlas Hive hook and data lineage troubleshooting

Explorer

I run Atlas 0.8 on HDInsight 3.5 and would like to take advantage of Hive Hook. I've made all the necessary configurations as described in the documentation. http://atlas.apache.org/0.8.0-incubating/Bridge-Hive.html . Then I follow a simple tutorial and create two table brancha and branchb and third one CTAS branch_intersect as a result of a join between those two. https://community.hortonworks.com/articles/36121/using-apache-atlas-to-view-data-lineage.html

Anyhow, I expect the tables to show up in the catalog but they don't. My question is how do I troubleshoot that kind of issue? Where to look?

When I execute import-hive.sh, the tables are imported but without data lineage. Does anyone have a clue on how to troubleshoot those issues?

Any ideas appreciated:)

5 REPLIES 5

Explorer

Ok, I've got this exception, AtlasBaseException after running import-hive.sh

2017-08-22 18:35:07,021 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ graph rollback due to exception AtlasBaseException:Instance hive_table with unique attribute {qualifiedName=default.branch_intersect3@primary} does not exist (GraphTransactionInterceptor:64) 2017-08-22 18:35:07,022 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ Cannot find entity with type: hive_table, attribute: qualifiedName and value: default.branch_intersect3@primary (EntityResource:798) 2017-08-22 18:35:07,022 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ Unable to get instance definition for type=hive_table, qualifiedName=default.branch_intersect3@primary (EntityResource:832) javax.ws.rs.WebApplicationException at org.apache.atlas.web.resources.EntityResource.toWebApplicationException(EntityResource.java:1201) at org.apache.atlas.web.resources.EntityResource.getEntityDefinitionByAttribute(EntityResource.java:799) at org.apache.atlas.web.resources.EntityResource.getEntity(EntityResource.java:762) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

Cloudera Employee

I had same issue and it has been resolved by adding extra kafka brokers (I had one and made it 3). Big thanks to @Ronak bansal for the solution "Either install 3 kafka brokers or change offsets.topic.replication.factor=1 in kafka and restart kafka + atlas"

Explorer

Hi,

I am facing the same issue.

Data are imported but no lineage is found even if it should appear.

 

Any hint about the solution?

 

Thanks,

Daniele.

Community Manager

@dansteu, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.


Regards,

Vidya Sargur,
Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Explorer

Hi,

I created yesterday also this new thread https://community.cloudera.com/t5/Support-Questions/Data-Lineage-not-shown-after-importing-Hive-enti...

 

Thanks in advance for support.

 

Best Regards,

Daniele.