https://community.cloudera.com/t5/Archives-of-Support-Questions/java-io-IOException-Cannot-find-AWS-access-key/m-p/229596#M64067 <A rel="user" href="https://community.cloudera.com/users/17949/rakanchi.html" nodeid="17949">@rakanchi</A><P>It is a known bug in Multi-threaded access to CredentialProviderFactory is not thread-safe. I had similar case with one of the customer. Had to apply hdfs patch : HADOOP-14195</P> Fri, 30 Jun 2017 20:03:59 GMT nyadav 2017-06-30T20:03:59Z https://community.cloudera.com/t5/Archives-of-Support-Questions/java-io-IOException-Cannot-find-AWS-access-key/m-p/229595#M64066 <P>I've a Spark job copying file from S3 bucket to HDFS using spark data frame api. </P><P>if we change --executor-cores to 1. then the job is successfully completing without any issues. if we change --executor-cores to 5 then it is throwing below error. </P><P>java.io.IOException: Cannot find AWS access key. </P><P>at org.apache.hadoop.fs.s3a.S3AFileSystem.getAWSAccessKeys(S3AFileSystem.java:358)</P> Fri, 30 Jun 2017 19:09:53 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/java-io-IOException-Cannot-find-AWS-access-key/m-p/229595#M64066 rakanchi 2017-06-30T19:09:53Z https://community.cloudera.com/t5/Archives-of-Support-Questions/java-io-IOException-Cannot-find-AWS-access-key/m-p/229596#M64067 <A rel="user" href="https://community.cloudera.com/users/17949/rakanchi.html" nodeid="17949">@rakanchi</A><P>It is a known bug in Multi-threaded access to CredentialProviderFactory is not thread-safe. I had similar case with one of the customer. Had to apply hdfs patch : HADOOP-14195</P> Fri, 30 Jun 2017 20:03:59 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/java-io-IOException-Cannot-find-AWS-access-key/m-p/229596#M64067 nyadav 2017-06-30T20:03:59Z