Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH5.4.7 - Cannot access s3a:// objects

Highlighted

CDH5.4.7 - Cannot access s3a:// objects

Hello,

 

As of this morning, attempts to access data via the s3a:// schema are failing with the following exception:

 

com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 5B6173042348847B), S3 Extended Request ID: dEaSmjayNYR5jlzZnHgFfmEjUnPFdhky5aHaTXs39xOnCR22v7xX7QQQ9NPLbJR/
	at com.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:820)
	at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:439)
	at com.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:245)
	at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3722)
	at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3675)
	at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:620)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:919)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:76)
	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)

This seems to be related to this issue: https://github.com/aws/aws-sdk-java/issues/444. My project explicitly uses joda-time 2.8.2, which is meant to fix this issue, but I think it's occuring despite this due to the com.cloudera.org.joda.time... classes included in your version of hadoop-aws.jar. I can workaround by using s3n:// or a JRE prior to 8u60. I believe this is specific to 5.4.7, as it has only popped up since we upgraded from 5.4.4. Any ideas?