Support Questions
Find answers, ask questions, and share your expertise

Hive metadata import to atlas gets failed

Highlighted

Hive metadata import to atlas gets failed

Contributor

Hi ,

I tired running import.sh present in atlas-pkg/hook-bin to import hive metadata to atlas.

But it got failed with following exception:

Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API UPDATE_ENTITY_PARTIAL failed with status 400 (Bad Request) Response Body ({"error":"java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while requesting metadata from brokers for ATLAS_ENTITIES-0",............................

....................................................................

Failed to import Hive Data Model!!!

Please advice a solution ASAP.

6 REPLIES 6
Highlighted

Re: Hive metadata import to atlas gets failed

Contributor

can you check if kafka service is running and the topic ATLAS_ENTITIES is created ?

Highlighted

Re: Hive metadata import to atlas gets failed

Contributor

Yes, Kafka listed the topic but with a warning.

[root@hdp-dn02 bin]# ./kafka-topics.sh --list --zookeeper <ip>:2181

[2017-02-06 19:18:04,781] WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/usr/hdp/current/kafka-broker/config/kafka_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. (org.apache.zookeeper.ClientCnxn)

ATLAS_ENTITIES

ATLAS_HOOK

ambari_kafka_service_check

The cluster was kerberised earlier and now it is disabled.Hence I removed the contents from the configured jaas.conf.

Highlighted

Re: Hive metadata import to atlas gets failed

@Vandana K R

Could you please provide more details on the issue?

  1. What is your HDP version?
  2. Is your cluster kerberized?
  3. Can you please paste the complete command used for import-hive.sh?
  4. Also, can you attach the atlas application log?

The above details are needed for further debugging.

Highlighted

Re: Hive metadata import to atlas gets failed

@Vandana K R

You can also refer to my article on importing hive metadata to atlas.

https://community.hortonworks.com/articles/61274/import-hive-metadata-into-atlas.html

Re: Hive metadata import to atlas gets failed

Contributor
  1. What is your HDP version? ==> 2.5.3.0
  2. Is your cluster kerberized? ==> It is not kerberised now.
  3. Can you please paste the complete command used for import-hive.sh? ==>./usr/hdp/2.5.3.0-37/atlas/hook-bin/import-hive.sh
  4. Also, can you attach the atlas application log?
Highlighted

Re: Hive metadata import to atlas gets failed

Explorer

Was there any solution for the above issue! Atlas was working fine initially but after kerberos was disabled I am neither able to import the hive data or see the table being created in Atlas