how to update core-site.xml across the whole cluster?

[#1]

I would like to update the core-site xml and replicate the changes across the whole cluster. What's the usual way to do that in Cloudera?

 Is the safty value configuration on hdfs service the only option? if yes, could you should an example how to fill in the web console? just copy the entire property portion for update?

[#2 · Accepted solution]

Assuming you are using CM 5.0 or higher, your Cloudera Manager home page should indicate that you have stale client configuration.

/etc/hadoop/conf is only updated by deploying client configuration. Until you do so, your changes won't be reflected in those files.

Note that roles like NameNode and TaskTracker don't use /etc/hadoop/conf, and don't update files like /etc/hadoop/conf/core-site.xml when those roles are refreshed or restarted. Roles and commands use configuration in private subdirectories of /var/run/cloudera-scm-agent/process/

Thanks,

Darren

[#3]

Yes, you should modify the hdfs configuration for core-site.xml.

Safety valves are documented here:

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Managing-...

You'll need the whole XML snippet

<property>
<name>...</name>
<value>...</value>
</property>

Thanks,

Darren

[#4]

I added several properties in the textbox of "Cluster-wide Advanced Configuration Snippet (Safety Value) for core-site.xml", after restarting hdfs services, the new properties are not in core-site.xml in any of hosts.

I checked core-site.xml in  /etc/hadoop/conf/core-site.xml at each host, there is no change; while the core-site file from "Download Client configuraiton" reflects the changes. What's wrong with it?

[#5 · Accepted solution]

Assuming you are using CM 5.0 or higher, your Cloudera Manager home page should indicate that you have stale client configuration.

/etc/hadoop/conf is only updated by deploying client configuration. Until you do so, your changes won't be reflected in those files.

Note that roles like NameNode and TaskTracker don't use /etc/hadoop/conf, and don't update files like /etc/hadoop/conf/core-site.xml when those roles are refreshed or restarted. Roles and commands use configuration in private subdirectories of /var/run/cloudera-scm-agent/process/

Thanks,

Darren

[#6 · Accepted solution]

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Managing-...

From the above link you set it Service-Wide the replication which will pickup at runtime after cluster restart, , no need to update in core-site.xml