Reply
Explorer
Posts: 8
Registered: ‎09-14-2015

Unable to create hive table s3a as location using jceks file

Hi Guys,

 

Created jceks file with aws credentials and configured fs.s3a.security.credential.provider.path=<jceks file path> in core-site.xml, whitelisted fs.s3a.security.credential.provider.path in hive-site.xml.

 

Able to performs hdfs cmds on s3a buckets.

 

Getting below error while creating hive table with s3a location

 

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.io.IOException Cannot find password option fs.s3a.access.key) (state=08S01,code=1)

 

Create table syntax: create table s3a_test1(id int) location 's3a://bucketname/user/dev/'

 

Tested Environments: CDH 5.13.x, CDH 5.14.x, CDH 5.15.x & CDH 6.0(Beta)

 

Note: We tried with regular s3a config paramets(fs.s3a) and multi bucketing paramets(fs.s3a.bucket.bucketname) also.

Cloudera Employee
Posts: 577
Registered: ‎03-23-2015

Re: Unable to create hive table s3a as location using jceks file

The error stated that:

Cannot find password option fs.s3a.access.key

have you checked that this configuration is set in hive-site.xml?
Explorer
Posts: 8
Registered: ‎09-14-2015

Re: Unable to create hive table s3a as location using jceks file

Yep!

 

I have set jceks configuration in hive and tried to create hive table but same error...

 

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.io.IOException Cannot find password option fs.s3a.access.key) (state=08S01,code=1)

 

FYI: Able to create table with same configuration in HDP 3.0

 

But our goal is to make work on Cloudera only, since 90% of our clients using Cloudera CDH!

 

Thank you!

ram

Announcements