Support Questions
Find answers, ask questions, and share your expertise

Unable to create hive table s3a as location using jceks file

Highlighted

Unable to create hive table s3a as location using jceks file

Explorer

Hi Guys,

 

Created jceks file with aws credentials and configured fs.s3a.security.credential.provider.path=<jceks file path> in core-site.xml, whitelisted fs.s3a.security.credential.provider.path in hive-site.xml.

 

Able to performs hdfs cmds on s3a buckets.

 

Getting below error while creating hive table with s3a location

 

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.io.IOException Cannot find password option fs.s3a.access.key) (state=08S01,code=1)

 

Create table syntax: create table s3a_test1(id int) location 's3a://bucketname/user/dev/'

 

Tested Environments: CDH 5.13.x, CDH 5.14.x, CDH 5.15.x & CDH 6.0(Beta)

 

Note: We tried with regular s3a config paramets(fs.s3a) and multi bucketing paramets(fs.s3a.bucket.bucketname) also.

2 REPLIES 2
Highlighted

Re: Unable to create hive table s3a as location using jceks file

Guru
The error stated that:

Cannot find password option fs.s3a.access.key

have you checked that this configuration is set in hive-site.xml?
Highlighted

Re: Unable to create hive table s3a as location using jceks file

Explorer

Yep!

 

I have set jceks configuration in hive and tried to create hive table but same error...

 

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.io.IOException Cannot find password option fs.s3a.access.key) (state=08S01,code=1)

 

FYI: Able to create table with same configuration in HDP 3.0

 

But our goal is to make work on Cloudera only, since 90% of our clients using Cloudera CDH!

 

Thank you!

ram