Support Questions
Find answers, ask questions, and share your expertise

tutorial 1 metastore connection failed

New Contributor

I download the VM image for virtual box version 5.5.0.0

I start cloudera express.  Eventually all starts successfully and I am given a hit ENTER to exit.

I leave that...

 

I start another terminal to run tutoral 1 and run this command:

sqoop import-all-tables -m 1 --connect jdbc:mysql://quickstart:3306/retail_db --username=retail_dba --password=cloudera --compression-codec=snappy --as-parquetfile --warehouse-dir=/user/hive/warehouse --hive-import

 

And I get this back:

Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/03/02 09:29:55 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.0
16/03/02 09:29:55 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/03/02 09:29:55 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
16/03/02 09:29:55 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
16/03/02 09:29:55 WARN tool.BaseSqoopTool: It seems that you're doing hive import directly into default
16/03/02 09:29:55 WARN tool.BaseSqoopTool: hive warehouse directory which is not supported. Sqoop is
16/03/02 09:29:55 WARN tool.BaseSqoopTool: firstly importing data into separate directory and then
16/03/02 09:29:55 WARN tool.BaseSqoopTool: inserting data into hive. Please consider removing
16/03/02 09:29:55 WARN tool.BaseSqoopTool: --target-dir or --warehouse-dir into /user/hive/warehouse in
16/03/02 09:29:55 WARN tool.BaseSqoopTool: case that you will detect any issues.
16/03/02 09:29:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/03/02 09:29:55 INFO tool.CodeGenTool: Beginning code generation
16/03/02 09:29:55 INFO tool.CodeGenTool: Will generate java class as codegen_categories
16/03/02 09:29:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/03/02 09:29:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/03/02 09:29:55 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-cloudera/compile/8328556a3098f37571a8d928046f4ed8/codegen_categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/03/02 09:29:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/8328556a3098f37571a8d928046f4ed8/codegen_categories.jar
16/03/02 09:29:57 WARN manager.MySQLManager: It looks like you are importing from mysql.
16/03/02 09:29:57 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
16/03/02 09:29:57 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
16/03/02 09:29:57 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
16/03/02 09:29:57 INFO mapreduce.ImportJobBase: Beginning import of categories
16/03/02 09:29:57 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/03/02 09:29:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/03/02 09:29:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
16/03/02 09:30:00 INFO hive.metastore: Trying to connect to metastore with URI thrift://quickstart.cloudera:9083
16/03/02 09:30:00 WARN hive.metastore: Failed to connect to the MetaStore Server...
16/03/02 09:30:00 INFO hive.metastore: Waiting 1 seconds before next connection attempt.
16/03/02 09:30:01 INFO hive.metastore: Trying to connect to metastore with URI thrift://quickstart.cloudera:9083
16/03/02 09:30:01 WARN hive.metastore: Failed to connect to the MetaStore Server...
16/03/02 09:30:01 INFO hive.metastore: Waiting 1 seconds before next connection attempt.
16/03/02 09:30:02 INFO hive.metastore: Trying to connect to metastore with URI thrift://quickstart.cloudera:9083
16/03/02 09:30:02 WARN hive.metastore: Failed to connect to the MetaStore Server...
16/03/02 09:30:02 INFO hive.metastore: Waiting 1 seconds before next connection attempt.
16/03/02 09:30:03 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetOperationException: Hive metastore exception
org.kitesdk.data.DatasetOperationException: Hive metastore exception
at org.kitesdk.data.spi.hive.MetaStoreUtil.<init>(MetaStoreUtil.java:135)
at org.kitesdk.data.spi.hive.MetaStoreUtil.get(MetaStoreUtil.java:101)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.getMetaStoreUtil(HiveAbstractMetadataProvider.java:63)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.resolveNamespace(HiveAbstractMetadataProvider.java:270)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.resolveNamespace(HiveAbstractMetadataProvider.java:255)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.exists(HiveAbstractMetadataProvider.java:159)
at org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository.exists(FileSystemDatasetRepository.java:262)
at org.kitesdk.data.Datasets.exists(Datasets.java:629)
at org.kitesdk.data.Datasets.exists(Datasets.java:646)
at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:117)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:260)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:111)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:419)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:234)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:179)
at org.kitesdk.data.spi.hive.MetaStoreUtil.<init>(MetaStoreUtil.java:133)
at org.kitesdk.data.spi.hive.MetaStoreUtil.get(MetaStoreUtil.java:101)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.getMetaStoreUtil(HiveAbstractMetadataProvider.java:63)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.resolveNamespace(HiveAbstractMetadataProvider.java:270)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.resolveNamespace(HiveAbstractMetadataProvider.java:255)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.exists(HiveAbstractMetadataProvider.java:159)
at org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository.exists(FileSystemDatasetRepository.java:262)
at org.kitesdk.data.Datasets.exists(Datasets.java:629)
at org.kitesdk.data.Datasets.exists(Datasets.java:646)
at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:117)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:260)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:111)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 24 more
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:466)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:234)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:179)
at org.kitesdk.data.spi.hive.MetaStoreUtil.<init>(MetaStoreUtil.java:133)
... 20 more

 

---------------------------------------------

 

I read somewhere that metastore the service was not started so I tried to start that last time I loaded

this with:

hive --service metastore 

 

It came back and said it was starting but then nothing for a really long time.  So I gave up and restarted and

reloaded everything to try again.  

 

Running a 4 core vm with 10240 MB memory.

 

Any ideas?