Created on 08-27-2018 08:40 AM - edited 09-16-2022 06:38 AM
Hi Guys,
Created jceks file with aws credentials and configured fs.s3a.security.credential.provider.path=<jceks file path> in core-site.xml, whitelisted fs.s3a.security.credential.provider.path in hive-site.xml.
Able to performs hdfs cmds on s3a buckets.
Getting below error while creating hive table with s3a location
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.io.IOException Cannot find password option fs.s3a.access.key) (state=08S01,code=1)
Create table syntax: create table s3a_test1(id int) location 's3a://bucketname/user/dev/'
Tested Environments: CDH 5.13.x, CDH 5.14.x, CDH 5.15.x & CDH 6.0(Beta)
Note: We tried with regular s3a config paramets(fs.s3a) and multi bucketing paramets(fs.s3a.bucket.bucketname) also.
Created 08-31-2018 07:27 PM
Created 09-01-2018 08:31 AM
Yep!
I have set jceks configuration in hive and tried to create hive table but same error...
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.io.IOException Cannot find password option fs.s3a.access.key) (state=08S01,code=1)
FYI: Able to create table with same configuration in HDP 3.0
But our goal is to make work on Cloudera only, since 90% of our clients using Cloudera CDH!
Thank you!
ram