Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hbase configuration hbase-site.xml command line

avatar
Explorer

Hi,

 

I am trying to implement a method to optimize Hbase with machine learning.

I have to create a dataset for that I have to change severals time the configuration of Hbase

With cloudera manager it is possible via UI but for my data of 250 configuration it's to long to change manually the configuration.

 

How can I modify the configuration of Hbase automatically ? 

I have a xml file (or json) with the new configuration but I don't find how to upload it to Hbase and Cloudera manager.

 

How can I upload it ? 

 

Thanks

1 ACCEPTED SOLUTION

avatar
Master Guru

@yassine24,

 

This shows how to update a service configuration

http://cloudera.github.io/cm_api/docs/python-client/#configuring-services-and-roles

 

You need to update the config with the attribute and value.

The configuration is JSON format, but the safety valve you want is in XML format.

 

An example of how to update a safety valve (hdfs in this case) via REST API is here:

curl -iv -X PUT -H "Content-Type:application/json" -H "Accept:application/json" -d '{"items":[{ "name": "core_site_safety_valve","value": "<property><name>hadoop.proxyuser.ztsps.users</name><value>*</value></property><property><name>hadoop.proxyuser.ztsps.groups</name><value>*</value></property>"}]}' http://admin:admin@10.1.0.1:7180/api/v12/clusters/cluster/services/hdfs/config

 

I am pretty sure you can pass the JSON as shown above in the -d argument hbase.update_config() or whatever

View solution in original post

6 REPLIES 6

avatar
Super Collaborator

avatar
Explorer

I already used the API.

 

My problem is that I generated a hbase-site.xml with new value for severals parameters.

Now I want to import it into Cloudera Manager with API. 

So I am looking for a way to put the xml file into Cloudera Manager without using the Cloudera manager UI.

 

With the Api I succed to retrieve the attributes : "hbase_service_config_safety_valve". But I don't find how can I modify it in my python code. 

 

Is there a way to put some xml code directly into Cloudera Manager without GUI ?

 

Thanks 

avatar
Super Collaborator

From command line I don't know if there is a way to upload xml for a specific service. You can do it for the cluster.

But you should check

https://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_services_-serviceName-_co...

and

https://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_services_-serviceName-_ro...

You can also modify parameters by using PUT instead of GET

avatar
Explorer
Can you explain me what I have to use for upload xml for the cluster ?

Thank you

avatar
Super Collaborator

My mistake. Cloudera works with json not xml.

Anyhow, to upload complete cluster config you can do it by following https://gist.github.com/gwenshap/7044525 but this is for serious cases.

On the other hand, you can push only HBase configuration with the previous links I send you. You just have to prepare config in json format.

avatar
Master Guru

@yassine24,

 

This shows how to update a service configuration

http://cloudera.github.io/cm_api/docs/python-client/#configuring-services-and-roles

 

You need to update the config with the attribute and value.

The configuration is JSON format, but the safety valve you want is in XML format.

 

An example of how to update a safety valve (hdfs in this case) via REST API is here:

curl -iv -X PUT -H "Content-Type:application/json" -H "Accept:application/json" -d '{"items":[{ "name": "core_site_safety_valve","value": "<property><name>hadoop.proxyuser.ztsps.users</name><value>*</value></property><property><name>hadoop.proxyuser.ztsps.groups</name><value>*</value></property>"}]}' http://admin:admin@10.1.0.1:7180/api/v12/clusters/cluster/services/hdfs/config

 

I am pretty sure you can pass the JSON as shown above in the -d argument hbase.update_config() or whatever