Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Enabled S3Guard in CDH 5.16.2 cluster but still seeing S3 eventual consistency issues

Enabled S3Guard in CDH 5.16.2 cluster but still seeing S3 eventual consistency issues

Contributor

Hi All,

 

We were facing issues when we're writing data to S3 due to eventual consistency problem. Later found `S3Guard` helps to fix this issue.

 

We've enabled S3Guard in our cluster, but still we're seeing job failures.

 

I have verified data in DynamoDB, and it definitely writes the data, and also I see that we also have reads for DynamoDB.

 

May I know is there a way in CDH cluster to enable logs to test if every request is going through DynamoDB and log the status of DynamoDB write/read? If there is a way to enable logging may I know how to enable it and check the logs?

 

Please note that we've added S3Guard into our cluster using IAM role based authentication, with all IAM roles that required access, also added S3connectivity only for our master roles.

 

Nagamalleswara_0-1594134882171.png

 

Don't have an account?
Coming from Hortonworks? Activate your account here