06-08-2018 10:56 AM
I am looking to add the following properties:
<property> <name>fs.s3a.access.key</name> <value>Amazon S3 Access Key</value> </property> <property> <name>fs.s3a.secret.key</name> <value>Amazon S3 Secret Key</value> </property>
to the file /etc/hive/conf.cloudera.hive/core-site.xml Now I am adding them manually to the file but if the changes are pushed looks like Cloudera Manager disregards the information already present in the file.
06-08-2018 11:55 PM
When the cluster is managed by Cloudera manager , its bad approach to manually edit the local file .
Cloudera manager by design will not pick up the changes .
You should push the changes using Cloudera manager web ui -> hive ->configuration -> advace config / safety valve . it would should us stale and restart of the service would be needed .
Hope this helps .
06-09-2018 01:51 AM - edited 06-09-2018 01:54 AM
Pls refer the below link, it will explain
06-11-2018 01:44 PM
06-11-2018 01:46 PM
I did configure the s3 connector. But that is not allowing Hive to Read Write. The blog asks to update the core-site.xml and keep a local copy but I need it for all users.
06-12-2018 05:46 AM
1. What is your Cloudera Version? If you are using Cloudera 5.10 or above then you can follow the instruction from the link that i've given above espectially 'Adding AWS Credentials' . You didn't mentioned whether you have already tried or not
2. Not sure you have restarted your cluster after the configuration change
3. If you follow this option, it will be applicable for all the users
4. I don't know which blog that you are following and how old it is... if you are using cloudera then use cloudera document
06-12-2018 03:02 PM
Thank you for your Inputs.
06-13-2018 05:31 AM
The link that you have provided says "If you have selected IAM authentication, no additional steps are needed" - which includes editing the core-site.xml
You have confirmed that added "AWS Credentials" in "External Accounts" from "Administration" menu - which is IAM role based authentication.
so you don't need to do both