on: Cloudera Manager 4.8.4 I set the fs.s3n.awsSecretAccessKey property in the core-site.xml through the HDFS safety valve, and still can't run the job. Running the job gives me this error: java.lang.IllegalArgumentException: AWS Secret Access Key must be specified as the password of a s3n URL, or by setting the fs.s3n.awsSecretAccessKey property. Meanwhile, I can run the job with the same credentials on a different cluster. Everything I've read says once you add these credentials, deploy the config and restart the cluster, you're good to go. I've also tried adding the property to the hdfs-site.xml and I've compared the jar file on both clusters to make sure they md5sum the same. Anyone run into some issues with their AWS credentials?
... View more
I just saw this after an office move. The servers were offline for a while. This is what I did for our CentOS Linux servers (similar, but not the same, steps would work for Ubuntu/Debian): * verified/updated the /etc/ntp.conf file to be the same across the cluster * stopped the ntp daemon: /etc/init.d/ntpd stop * connected to the ntp server named in the ntp.conf file: ntpupdate <ntp_server> * started the ntp daemon: /etc/init.d/ntpd start * added ntpd to the startup manager: chkconfig add ntpd * set ntpd to start in default reunlevels: chkconfig ntpd on This cleared the offset issue. If this doesn't do it for you, you might also have to look into drift, or dig deeper into the time synchronization.
... View more