Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎01-13-2018

Error during tutorial 1 CDH DatasetOperationException Hive MetaStore quickstart,cloudera:8020 failed

[ Edited ]

Hi,

 

I am trying running cloudera quickstart VM 5.12 virtualbox (on windows 10) and getting the following error:

 

sqoop import-all-tables \
> -m 1 \
> --connect jdbc:mysql://quickstart:3306/retail_db \
> --username=retail_dba \
> --password=cloudera \
> --compression-codec=snappy \
> --as-parquetfile \
> --warehouse-dir=/user/hive/warehouse \
> --hive-import
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/01/13 21:23:31 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.12.0
18/01/13 21:23:31 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/01/13 21:23:31 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
18/01/13 21:23:31 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
18/01/13 21:23:31 WARN tool.BaseSqoopTool: It seems that you're doing hive import directly into default
18/01/13 21:23:31 WARN tool.BaseSqoopTool: hive warehouse directory which is not supported. Sqoop is
18/01/13 21:23:31 WARN tool.BaseSqoopTool: firstly importing data into separate directory and then
18/01/13 21:23:31 WARN tool.BaseSqoopTool: inserting data into hive. Please consider removing
18/01/13 21:23:31 WARN tool.BaseSqoopTool: --target-dir or --warehouse-dir into /user/hive/warehouse in
18/01/13 21:23:31 WARN tool.BaseSqoopTool: case that you will detect any issues.
18/01/13 21:23:32 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/01/13 21:23:32 INFO tool.CodeGenTool: Beginning code generation
18/01/13 21:23:32 INFO tool.CodeGenTool: Will generate java class as codegen_categories
18/01/13 21:23:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
18/01/13 21:23:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
18/01/13 21:23:32 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/8817f3265d374ec9b5a91e9003581c58/codegen_categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/01/13 21:23:35 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/8817f3265d374ec9b5a91e9003581c58/codegen_categories.jar
18/01/13 21:23:35 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/01/13 21:23:35 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/01/13 21:23:35 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/01/13 21:23:35 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/01/13 21:23:35 INFO mapreduce.ImportJobBase: Beginning import of categories
18/01/13 21:23:35 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
18/01/13 21:23:35 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/01/13 21:23:36 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
18/01/13 21:23:36 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
18/01/13 21:23:37 INFO hive.metastore: Trying to connect to metastore with URI thrift://127.0.0.1:9083
18/01/13 21:23:37 INFO hive.metastore: Opened a connection to metastore, current connections: 1
18/01/13 21:23:37 INFO hive.metastore: Connected to metastore.
18/01/13 21:23:38 INFO hive.metastore: Closed a connection to metastore, current connections: 0
18/01/13 21:23:38 INFO hive.metastore: Trying to connect to metastore with URI thrift://127.0.0.1:9083
18/01/13 21:23:38 INFO hive.metastore: Opened a connection to metastore, current connections: 1
18/01/13 21:23:38 INFO hive.metastore: Connected to metastore.
18/01/13 21:23:38 INFO hive.metastore: Closed a connection to metastore, current connections: 0
18/01/13 21:23:38 INFO hive.metastore: Trying to connect to metastore with URI thrift://127.0.0.1:9083
18/01/13 21:23:38 INFO hive.metastore: Opened a connection to metastore, current connections: 1
18/01/13 21:23:38 INFO hive.metastore: Connected to metastore.
18/01/13 21:23:38 INFO hive.metastore: Closed a connection to metastore, current connections: 0
18/01/13 21:23:38 INFO hive.metastore: Trying to connect to metastore with URI thrift://127.0.0.1:9083
18/01/13 21:23:38 INFO hive.metastore: Opened a connection to metastore, current connections: 1
18/01/13 21:23:38 INFO hive.metastore: Connected to metastore.
18/01/13 21:23:38 INFO hive.HiveManagedMetadataProvider: Creating a managed Hive table named: categories
18/01/13 21:23:38 INFO hive.metastore: Closed a connection to metastore, current connections: 0
18/01/13 21:23:38 INFO hive.metastore: Trying to connect to metastore with URI thrift://127.0.0.1:9083
18/01/13 21:23:38 INFO hive.metastore: Opened a connection to metastore, current connections: 1
18/01/13 21:23:38 INFO hive.metastore: Connected to metastore.
18/01/13 21:23:40 INFO hive.metastore: Closed a connection to metastore, current connections: 0
18/01/13 21:23:40 INFO hive.metastore: Trying to connect to metastore with URI thrift://127.0.0.1:9083
18/01/13 21:23:40 INFO hive.metastore: Opened a connection to metastore, current connections: 1
18/01/13 21:23:40 INFO hive.metastore: Connected to metastore.
18/01/13 21:23:40 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetOperationException: Hive MetaStore exception
org.kitesdk.data.DatasetOperationException: Hive MetaStore exception
at org.kitesdk.data.spi.hive.MetaStoreUtil.createTable(MetaStoreUtil.java:252)
at org.kitesdk.data.spi.hive.HiveManagedMetadataProvider.create(HiveManagedMetadataProvider.java:87)
at org.kitesdk.data.spi.hive.HiveManagedDatasetRepository.create(HiveManagedDatasetRepository.java:81)
at org.kitesdk.data.Datasets.create(Datasets.java:239)
at org.kitesdk.data.Datasets.create(Datasets.java:307)
at org.apache.sqoop.mapreduce.ParquetJob.createDataset(ParquetJob.java:156)
at org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:130)
at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:130)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:267)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:513)
at org.apache.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:110)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: MetaException(message:Got exception: java.net.ConnectException Call From quickstart.cloudera/10.0.2.15 to quickstart.cloudera:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_with_environment_context_result$create_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:30374)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_with_environment_context_result$create_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:30342)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_with_environment_context_result.read(ThriftHiveMetastore.java:30268)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table_with_environment_context(ThriftHiveMetastore.java:1091)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table_with_environment_context(ThriftHiveMetastore.java:1077)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:2152)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:732)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:720)
at org.kitesdk.data.spi.hive.MetaStoreUtil$4.call(MetaStoreUtil.java:230)
at org.kitesdk.data.spi.hive.MetaStoreUtil$4.call(MetaStoreUtil.java:227)
at org.kitesdk.data.spi.hive.MetaStoreUtil.doWithRetry(MetaStoreUtil.java:70)
at org.kitesdk.data.spi.hive.MetaStoreUtil.createTable(MetaStoreUtil.java:242)
... 18 more

 

Any suggestions. Have looked into previously posted questions/solutions and tried but could not resolve.

 

Thanks

Arun

Contributor
Posts: 33
Registered: ‎02-15-2016

Re: Error during tutorial 1 CDH DatasetOperationException Hive MetaStore quickstart,cloudera:8020 fa

It looks like either the Hive Metastore server is not up, or the database that holds metastore is not up.

 

Please verify both of these. You could try connecting to Hive via beeline to find that out.

New Contributor
Posts: 1
Registered: ‎02-14-2018

Re: Error during tutorial 1 CDH DatasetOperationException Hive MetaStore quickstart,cloudera:8020 fa

Hi , I have the same issue too.

 

Can enlighten,

1) how to verify whether hive metastore server and database that holds metastore is up ?

2) how to conntect to hive via beeline ?

 

Many thanks , need your inputs to carry on my tutorial .

Announcements