Reply
Highlighted
Explorer
Posts: 10
Registered: ‎07-17-2018
Accepted Solution

Hbase configuration hbase-site.xml command line

Hi,

 

I am trying to implement a method to optimize Hbase with machine learning.

I have to create a dataset for that I have to change severals time the configuration of Hbase

With cloudera manager it is possible via UI but for my data of 250 configuration it's to long to change manually the configuration.

 

How can I modify the configuration of Hbase automatically ? 

I have a xml file (or json) with the new configuration but I don't find how to upload it to Hbase and Cloudera manager.

 

How can I upload it ? 

 

Thanks

Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: Hbase configuration hbase-site.xml command line

Explorer
Posts: 10
Registered: ‎07-17-2018

Re: Hbase configuration hbase-site.xml command line

I already used the API.

 

My problem is that I generated a hbase-site.xml with new value for severals parameters.

Now I want to import it into Cloudera Manager with API. 

So I am looking for a way to put the xml file into Cloudera Manager without using the Cloudera manager UI.

 

With the Api I succed to retrieve the attributes : "hbase_service_config_safety_valve". But I don't find how can I modify it in my python code. 

 

Is there a way to put some xml code directly into Cloudera Manager without GUI ?

 

Thanks 

Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: Hbase configuration hbase-site.xml command line

From command line I don't know if there is a way to upload xml for a specific service. You can do it for the cluster.

But you should check

https://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_services_-serviceName-_co...

and

https://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_services_-serviceName-_ro...

You can also modify parameters by using PUT instead of GET

Explorer
Posts: 10
Registered: ‎07-17-2018

Re: Hbase configuration hbase-site.xml command line

Can you explain me what I have to use for upload xml for the cluster ?

Thank you
Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: Hbase configuration hbase-site.xml command line

My mistake. Cloudera works with json not xml.

Anyhow, to upload complete cluster config you can do it by following https://gist.github.com/gwenshap/7044525 but this is for serious cases.

On the other hand, you can push only HBase configuration with the previous links I send you. You just have to prepare config in json format.

Posts: 934
Topics: 1
Kudos: 218
Solutions: 117
Registered: ‎04-22-2014

Re: Hbase configuration hbase-site.xml command line

@yassine24,

 

This shows how to update a service configuration

http://cloudera.github.io/cm_api/docs/python-client/#configuring-services-and-roles

 

You need to update the config with the attribute and value.

The configuration is JSON format, but the safety valve you want is in XML format.

 

An example of how to update a safety valve (hdfs in this case) via REST API is here:

curl -iv -X PUT -H "Content-Type:application/json" -H "Accept:application/json" -d '{"items":[{ "name": "core_site_safety_valve","value": "<property><name>hadoop.proxyuser.ztsps.users</name><value>*</value></property><property><name>hadoop.proxyuser.ztsps.groups</name><value>*</value></property>"}]}' http://admin:admin@10.1.0.1:7180/api/v12/clusters/cluster/services/hdfs/config

 

I am pretty sure you can pass the JSON as shown above in the -d argument hbase.update_config() or whatever

Announcements