Hi,   I am trying to import a hive table that has an MYSQL DB as metastore into Atlas by using import-hive.sh command but even when I explicitly pass the database name it fails to detect the same.   @Shelton, Can you please suggest on this?     The above snippet shows AMBDB existing in the MySQL DB. ... View more

Hi,    I am facing the below error when running the import-hive.sh to import hive metadata to Apache Atlas.    2020-08-08T07:41:25,369 ERROR [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Import failed com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19] at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:246) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findEntity(HiveMetaStoreBridge.java:780) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findDatabase(HiveMetaStoreBridge.java:745) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:399) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:281) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:251) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:168) [hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315] Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_112] at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_112] at java.net.SocketInputStream.read(SocketInputStream.java:170) ~[?:1.8.0_112] at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_112] at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_112] at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_112] at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_112] at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704) ~[?:1.8.0_112] at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) ~[?:1.8.0_112] at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569) ~[?:1.8.0_112] at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[?:1.8.0_112] at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_112] at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] ... 16 more Failed to import Hive Meta Data!!! ... View more

@Shelton@deepesh1,   I need your help in figuring out why this Hive Metastore and Hive Server 2 are not working. Please find the ambari-error.log below:   Traceback (most recent call last): File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 995, in restart self.status(env) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 87, in status check_process_status(status_params.hive_metastore_pid) File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/check_process_status.py", line 43, in check_process_status raise ComponentIsNotRunning() ComponentIsNotRunning The above exception was the cause of the following exception: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 201, in <module> HiveMetastore().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute method(env) File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 1006, in restart self.start(env, upgrade_type=upgrade_type) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 61, in start create_metastore_schema() # execute without config lock File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive.py", line 487, in create_metastore_schema user = params.hive_user File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run returns=self.resource.returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'export HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/ ; /usr/hdp/current/hive-server2/bin/schematool -initSchema -dbType mysql -userName ambari_user -passWord [PROTECTED] -verbose' returned 1. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Initializing the schema to: 3.1.1000 Metastore connection URL: jdbc:mysql://fgpoc.ambarisvr.com/ Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: ambari_user Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. Starting metastore schema initialization to 3.1.1000 Initialization script hive-schema-3.1.1000.mysql.sql Connecting to jdbc:mysql://fgpoc.ambarisvr.com/ Connected to: MySQL (version 5.7.31-0ubuntu0.16.04.1) Driver: MySQL Connector/J (version mysql-connector-java-8.0.21 (Revision: 33f65445a1bcc544eb0120491926484da168f199)) Transaction isolation: TRANSACTION_READ_COMMITTED 0: jdbc:mysql://fgpoc.ambarisvr.com/> !autocommit on Autocommit status: true 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@C HARACTER_SET_CLIENT */ No rows affected (0.01 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@ CHARACTER_SET_RESULTS */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_COLLATION_CONNECTION=@@C OLLATION_CONNECTION */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET NAMES utf8 */ No rows affected (0.002 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40103 SET TIME_ZONE='+00:00' */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_C HECKS, UNIQUE_CHECKS=0 */ No rows affected (0.002 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOR EIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL _MODE='NO_AUTO_VALUE_ON_ZERO' */ No rows affected (0 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, S QL_NOTES=0 */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @saved_cs_client = @@char acter_set_client */ No rows affected (0.005 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET character_set_client = utf8 * / No rows affected (0.002 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> CREATE TABLE IF NOT EXISTS `BUCKETING_COLS ` ( `SD_ID` bigint(20) NOT NULL, `BUCKET_COL_NAME` varchar(256) CHARACTER SET la tin1 COLLATE latin1_bin DEFAULT NULL, `INTEGER_IDX` int(11) NOT NULL, PRIMARY KE Y (`SD_ID`,`INTEGER_IDX`), KEY `BUCKETING_COLS_N49` (`SD_ID`), CONSTRAINT `BUCKE TING_COLS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 Error: No database selected (state=3D000,code=1046) Closing: 0: jdbc:mysql://fgpoc.ambarisvr.com/ Schema initialization FAILED! Metastore state would be inconsistent! Underlying cause: java.io.IOException : Schema script failed, errorcode 2 org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent! at org.apache.hadoop.hive.metastore.tools.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:66) at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.run(MetastoreSchemaTool.java:446) at org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) Caused by: java.io.IOException: Schema script failed, errorcode 2 at org.apache.hive.beeline.schematool.HiveSchemaTool.execSql(HiveSchemaTool.java:105) at org.apache.hive.beeline.schematool.HiveSchemaTool.execSql(HiveSchemaTool.java:83) at org.apache.hadoop.hive.metastore.tools.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:62) ... 8 more *** schemaTool failed ***   ... View more