Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

Solved Go to solution
Highlighted

Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

Contributor

when I study HDFS,I encounter one question,I want to make hadoop read the confighration file again,because I have changed the file, when stay hadoop open。and I don't want to reboot the hadoop service!

thank you very much!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

Hi @Xiong Duan - for most cases, in order to refresh the configurations in the running service, you have to restart the service and there isn't really a way around it.

However, if you have High Availability enabled (e.g. NameNode HA), there is a way you can refresh configs without downtime, but you must be very careful.

  1. If using Ambari - set up a Config Group with the relevant properties enabled. Add and remove the hosts as described below.
  2. Configure for the Standby NameNode
  3. Restart the Standby NN
  4. Setup the config for the Active NameNode
  5. Restart the Active NN. Failover will occur.
  6. Setup Config for a group of 2 DataNodes at a time and restart batches of 2 DN at a time

This may not be safe for all configuration properties - which properties do you want to refresh?

4 REPLIES 4

Re: Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

Guru

When you say "I want to make hadoop read the configuration file again..", do you mean Namenode service or any other service? Could you please clarify which service you want to refresh. Thanks.

Re: Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

Hi @Xiong Duan - for most cases, in order to refresh the configurations in the running service, you have to restart the service and there isn't really a way around it.

However, if you have High Availability enabled (e.g. NameNode HA), there is a way you can refresh configs without downtime, but you must be very careful.

  1. If using Ambari - set up a Config Group with the relevant properties enabled. Add and remove the hosts as described below.
  2. Configure for the Standby NameNode
  3. Restart the Standby NN
  4. Setup the config for the Active NameNode
  5. Restart the Active NN. Failover will occur.
  6. Setup Config for a group of 2 DataNodes at a time and restart batches of 2 DN at a time

This may not be safe for all configuration properties - which properties do you want to refresh?

Re: Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

Contributor

Thank you very much,I just want to change the replications of files that have upload to HDFS.So yesterdy,I found that one command can realize that.So thank you very much!

Re: Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

You're very welcome! Yes, for this the setrep command can change the replication factor for existing files and there is no need to change the value globally. For others' reference, the command is:

hdfs dfs -setrep [-R] [-w] <numReplicas> <path>
Don't have an account?
Coming from Hortonworks? Activate your account here