Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to download hdfs configuration files programatically?

Highlighted

How to download hdfs configuration files programatically?

New Contributor

I have a HDP cluster in HA mode & have java client that needs to download the configuration files (hdfs-site, core xml, etc) at runtime. How to achieve it? I believe cloudera manager provide URL way downloading config files, do we have something similar with ambari?

5 REPLIES 5

Re: How to download hdfs configuration files programatically?

Mentor

You can easily script that to read /etc/hadoop/conf and /etc/hive/conf. @Parmeet Arora

You can also query the Ambari database for configurations and you can use Ambari API to get the configs.

Re: How to download hdfs configuration files programatically?

New Contributor

@Artem Ervits to be more specific, i want to access config files over HTTP. Is there ambari api that exposes configs over http?

Re: How to download hdfs configuration files programatically?

Mentor

Here is an example how to do it with API. Link @Parmeet Arora

Re: How to download hdfs configuration files programatically?

@Parmeet Arora, can you please add more details about what your Java client needs to do end-to-end? Does it really need access to the raw XML files, or does it instead need access to a specific set of configuration values? Would this Java application run on one of the cluster nodes, or would it need to run external to the cluster? Would it possibly need to support multiple different clusters running with different configurations? These details might steer the solution in a few different directions. Thanks!

Re: How to download hdfs configuration files programatically?

you can go to /conf from the URL of any Hadoop service and get the conf file they used.

Be aware: these are the raw configurations; they may have things like hostnames as 0.0.0.0, telling the service to listen on all ports

Don't have an account?
Coming from Hortonworks? Activate your account here