Apache Atlas Hive hook and data lineage troubleshooting

[#1]

I run Atlas 0.8 on HDInsight 3.5 and would like to take advantage of Hive Hook. I've made all the necessary configurations as described in the documentation. http://atlas.apache.org/0.8.0-incubating/Bridge-Hive.html . Then I follow a simple tutorial and create two table brancha and branchb and third one CTAS branch_intersect as a result of a join between those two. https://community.hortonworks.com/articles/36121/using-apache-atlas-to-view-data-lineage.html

Anyhow, I expect the tables to show up in the catalog but they don't. My question is how do I troubleshoot that kind of issue? Where to look?

When I execute import-hive.sh, the tables are imported but without data lineage. Does anyone have a clue on how to troubleshoot those issues?

Any ideas appreciated:)

[#2]

Ok, I've got this exception, AtlasBaseException after running import-hive.sh

2017-08-22 18:35:07,021 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ graph rollback due to exception AtlasBaseException:Instance hive_table with unique attribute {qualifiedName=default.branch_intersect3@primary} does not exist (GraphTransactionInterceptor:64) 2017-08-22 18:35:07,022 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ Cannot find entity with type: hive_table, attribute: qualifiedName and value: default.branch_intersect3@primary (EntityResource:798) 2017-08-22 18:35:07,022 ERROR - [pool-1-thread-5 - db200196-5334-447e-ad81-4f44b14839df:] ~ Unable to get instance definition for type=hive_table, qualifiedName=default.branch_intersect3@primary (EntityResource:832) javax.ws.rs.WebApplicationException at org.apache.atlas.web.resources.EntityResource.toWebApplicationException(EntityResource.java:1201) at org.apache.atlas.web.resources.EntityResource.getEntityDefinitionByAttribute(EntityResource.java:799) at org.apache.atlas.web.resources.EntityResource.getEntity(EntityResource.java:762) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

[#3]

I had same issue and it has been resolved by adding extra kafka brokers (I had one and made it 3). Big thanks to @Ronak bansal for the solution "Either install 3 kafka brokers or change offsets.topic.replication.factor=1 in kafka and restart kafka + atlas"

[#4]

Hi,

I am facing the same issue.

Data are imported but no lineage is found even if it should appear.

Any hint about the solution?

Thanks,

Daniele.

[#5]

@dansteu, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.

[#6]

Hi,

I created yesterday also this new thread https://community.cloudera.com/t5/Support-Questions/Data-Lineage-not-shown-after-importing-Hive-enti...

Thanks in advance for support.

Best Regards,

Daniele.