Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hi,How to make hadoop to read the configuration file again,ex. hdfs-site.xml

avatar
Contributor

when I study HDFS,I encounter one question,I want to make hadoop read the confighration file again,because I have changed the file, when stay hadoop open。and I don't want to reboot the hadoop service!

thank you very much!

1 ACCEPTED SOLUTION

avatar

Hi @Xiong Duan - for most cases, in order to refresh the configurations in the running service, you have to restart the service and there isn't really a way around it.

However, if you have High Availability enabled (e.g. NameNode HA), there is a way you can refresh configs without downtime, but you must be very careful.

  1. If using Ambari - set up a Config Group with the relevant properties enabled. Add and remove the hosts as described below.
  2. Configure for the Standby NameNode
  3. Restart the Standby NN
  4. Setup the config for the Active NameNode
  5. Restart the Active NN. Failover will occur.
  6. Setup Config for a group of 2 DataNodes at a time and restart batches of 2 DN at a time

This may not be safe for all configuration properties - which properties do you want to refresh?

View solution in original post

4 REPLIES 4

avatar
Guru

When you say "I want to make hadoop read the configuration file again..", do you mean Namenode service or any other service? Could you please clarify which service you want to refresh. Thanks.

avatar

Hi @Xiong Duan - for most cases, in order to refresh the configurations in the running service, you have to restart the service and there isn't really a way around it.

However, if you have High Availability enabled (e.g. NameNode HA), there is a way you can refresh configs without downtime, but you must be very careful.

  1. If using Ambari - set up a Config Group with the relevant properties enabled. Add and remove the hosts as described below.
  2. Configure for the Standby NameNode
  3. Restart the Standby NN
  4. Setup the config for the Active NameNode
  5. Restart the Active NN. Failover will occur.
  6. Setup Config for a group of 2 DataNodes at a time and restart batches of 2 DN at a time

This may not be safe for all configuration properties - which properties do you want to refresh?

avatar
Contributor

Thank you very much,I just want to change the replications of files that have upload to HDFS.So yesterdy,I found that one command can realize that.So thank you very much!

avatar

You're very welcome! Yes, for this the setrep command can change the replication factor for existing files and there is no need to change the value globally. For others' reference, the command is:

hdfs dfs -setrep [-R] [-w] <numReplicas> <path>