Support Questions
Find answers, ask questions, and share your expertise

Error Hive creating external table in with S3a location

New Contributor

I am running Hadoop 3.3.1 and Hive 3.1.2 in Linux on Amazon Corretto 8 JDK on my local machine. I am trying to create an external table in S3 using the code below.

 

CREATE EXTERNAL TABLE IF NOT EXISTS user_orders(id int, user_id int, guid string, url_domain string, price int, city string, latitude decimal(10,5), longitude decimal(10,5), zip int, is_delivered boolean, note string)
COMMENT 'user orders details'
STORED AS PARQUET
LOCATION 's3a://[MyBucktName]/[Subfolder]/';

 

It fails with the following error:

 

2021-08-09T20:13:29,280 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] ql.Driver: Compiling command(queryId=ubuntu_20210809201329_06d5c799-2fec-4b07-905b-825886454956): CREATE EXTERNAL TABLE IF NOT EXISTS user_orders(id int, user_id int, guid string, url_domain string, price int, city string, latitude decimal(10,5), longitude decimal(10,5), zip int, is_delivered boolean, note string)
COMMENT 'user orders details'
STORED AS PARQUET
LOCATION 's3a://[MyBucktName]/[Subfolder]/'
2021-08-09T20:13:29,295 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://localhost:9083
2021-08-09T20:13:29,295 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 2
2021-08-09T20:13:29,296 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] metastore.HiveMetaStoreClient: Connected to metastore.
2021-08-09T20:13:29,296 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=ubuntu (auth:SIMPLE) retries=1 delay=1 lifetime=0
2021-08-09T20:13:29,302 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
2021-08-09T20:13:29,303 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] metastore.HiveMetaStoreClient: Closed a connection to metastore, current connections: 1
2021-08-09T20:13:29,303 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2021-08-09T20:13:29,303 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] parse.CalcitePlanner: Starting Semantic Analysis
2021-08-09T20:13:29,303 INFO [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] parse.CalcitePlanner: Creating table default.user_orders position=36
2021-08-09T20:13:29,333 ERROR [b66e2225-5d31-4069-8dd5-e90a0b034b40 main] ql.Driver: FAILED: SemanticException java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
org.apache.hadoop.hive.ql.parse.SemanticException: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.toReadEntity(BaseSemanticAnalyzer.java:2143)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.toReadEntity(BaseSemanticAnalyzer.java:2135)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.toReadEntity(BaseSemanticAnalyzer.java:2131)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:13093)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11974)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12129)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:330)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:659)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1826)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1773)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1768)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3467)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)

at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.tryQualifyPath(BaseSemanticAnalyzer.java:2153)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.toReadEntity(BaseSemanticAnalyzer.java:2140)

Caused by: java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
org/apache/hadoop/fs/s3a/Invoker.once(Ljava/lang/String;Ljava/lang/String;Lorg/apache/hadoop/util/functional/CallableRaisingIOE;)Ljava/lang/Object; @118: invokestatic
Reason:
Type 'com/amazonaws/AmazonClientException' (current frame, stack[2]) is not assignable to 'com/amazonaws/SdkBaseException'
locals: { 'java/lang/String', 'java/lang/String', 'org/apache/hadoop/util/functional/CallableRaisingIOE', 'com/amazonaws/AmazonClientException' }
stack: { 'java/lang/String', 'java/lang/String', 'com/amazonaws/AmazonClientException' }

 

 

I have failed to find a solution on the web. Any help will be greatly appreciated.

0 REPLIES 0