Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Apache Atlas Hive hook and data lineage troubleshooting

Explorer

I run Atlas 0.8 on HDInsight 3.5 and would like to take advantage of Hive Hook. I've made all the necessary configurations as described in the documentation. http://atlas.apache.org/0.8.0-incubating/Bridge-Hive.html . Then I follow a simple tutorial and create two table brancha and branchb and third one CTAS branch_intersect as a result of a join between those two. https://community.hortonworks.com/articles/36121/using-apache-atlas-to-view-data-lineage.html

Anyhow, I expect the tables to show up in the catalog but they don't. My question is how do I troubleshoot that kind of issue? Where to look?

When I execute import-hive.sh, the tables are imported but without data lineage. Does anyone have a clue on how to troubleshoot those issues?

Any ideas appreciated:)

5 REPLIES 5

Explorer

Ok, I've got this exception, AtlasBaseException after running import-hive.sh

2017-08-22 18:35:07,021 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ graph rollback due to exception AtlasBaseException:Instance hive_table with unique attribute {qualifiedName=default.branch_intersect3@primary} does not exist (GraphTransactionInterceptor:64) 2017-08-22 18:35:07,022 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ Cannot find entity with type: hive_table, attribute: qualifiedName and value: default.branch_intersect3@primary (EntityResource:798) 2017-08-22 18:35:07,022 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ Unable to get instance definition for type=hive_table, qualifiedName=default.branch_intersect3@primary (EntityResource:832) javax.ws.rs.WebApplicationException at org.apache.atlas.web.resources.EntityResource.toWebApplicationException(EntityResource.java:1201) at org.apache.atlas.web.resources.EntityResource.getEntityDefinitionByAttribute(EntityResource.java:799) at org.apache.atlas.web.resources.EntityResource.getEntity(EntityResource.java:762) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

Cloudera Employee

I had same issue and it has been resolved by adding extra kafka brokers (I had one and made it 3). Big thanks to @Ronak bansal for the solution "Either install 3 kafka brokers or change offsets.topic.replication.factor=1 in kafka and restart kafka + atlas"

Explorer

Hi,

I am facing the same issue.

Data are imported but no lineage is found even if it should appear.

 

Any hint about the solution?

 

Thanks,

Daniele.

Community Manager

@dansteu, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

Explorer

Hi,

I created yesterday also this new thread https://community.cloudera.com/t5/Support-Questions/Data-Lineage-not-shown-after-importing-Hive-enti...

 

Thanks in advance for support.

 

Best Regards,

Daniele.