We are getting below error when trying to read the table from s3. We are able to create the hive tables pointing to s3 bucket but while reading the table we are facing the issue.

[#1]

Note: We are not using accessKeyId and SecretKey in hadoop configuration. We have created a policy and attached it to s3 role.

hive> CREATE TABLE `global_param`(

`key` string,

`value` string)

ROW FORMAT DELIMITED

FIELDS TERMINATED BY '|'

STORED AS INPUTFORMAT

'org.apache.hadoop.mapred.TextInputFormat'

OUTPUTFORMAT

'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'

LOCATION

's3a:/XXXXXX/XXX/global_param'

TBLPROPERTIES (

'COLUMN_STATS_ACCURATE'='true',

'numFiles'='1',

'numRows'='0',

'rawDataSize'='0',

'totalSize'='1866',

'transient_lastDdlTime'='1463045355');

This returns OK.

hive> select * from global_param;

Failed with exception java.io.IOException:com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: 1AD9B28AF6F67613, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: i2sb9uE02euTHe16iJ+ZEa5BnVsGjUVh+sRigCnQpSTVF+qkATS/rQRst7SUqivikbRTBdUqCX4=

[#2]

Is it anon access or IAM policy?. Which version of HDP/Hadoop are you using?. Debug logs would provide details on which credential provider it is trying to use.

[#3]

@Shyam Shaw Is your cluster running on EC2 instances? What S3 access policy did you create?

[#4 · Accepted solution]

This issue has been resolved. AWS admin team was able to figure out the problem, they modified the policy and after that we were able to read the table from s3.

Thanks everyone for your help. 🙂

[#5]

Hi @Shyam Shaw Even I am facing the same error message, even though there are read and write permissions on the bucket. Are there any other permissions which needs to be added on Bucket?