I am looking to add the following properties:
<property> <name>fs.s3a.access.key</name> <value>Amazon S3 Access Key</value> </property> <property> <name>fs.s3a.secret.key</name> <value>Amazon S3 Secret Key</value> </property>
to the file /etc/hive/conf.cloudera.hive/core-site.xml Now I am adding them manually to the file but if the changes are pushed looks like Cloudera Manager disregards the information already present in the file.
When the cluster is managed by Cloudera manager , its bad approach to manually edit the local file .
Cloudera manager by design will not pick up the changes .
You should push the changes using Cloudera manager web ui -> hive ->configuration -> advace config / safety valve . it would should us stale and restart of the service would be needed .
Hope this helps .
Pls refer the below link, it will explain
I did configure the s3 connector. But that is not allowing Hive to Read Write. The blog asks to update the core-site.xml and keep a local copy but I need it for all users.
1. What is your Cloudera Version? If you are using Cloudera 5.10 or above then you can follow the instruction from the link that i've given above espectially 'Adding AWS Credentials' . You didn't mentioned whether you have already tried or not
2. Not sure you have restarted your cluster after the configuration change
3. If you follow this option, it will be applicable for all the users
4. I don't know which blog that you are following and how old it is... if you are using cloudera then use cloudera document
Thank you for your Inputs.
The link that you have provided says "If you have selected IAM authentication, no additional steps are needed" - which includes editing the core-site.xml
You have confirmed that added "AWS Credentials" in "External Accounts" from "Administration" menu - which is IAM role based authentication.
so you don't need to do both