question Re: Hbase configuration hbase-site.xml command line in Archives of Support Questions (Read Only)

Hbase configuration hbase-site.xml command line

yassine24 — Fri, 16 Sep 2022 13:28:30 GMT

Hi,

I am trying to implement a method to optimize Hbase with machine learning.

I have to create a dataset for that I have to change severals time the configuration of Hbase

With cloudera manager it is possible via UI but for my data of 250 configuration it's to long to change manually the configuration.

How can I modify the configuration of Hbase automatically ?

I have a xml file (or json) with the new configuration but I don't find how to upload it to Hbase and Cloudera manager.

How can I upload it ?

Thanks

Re: Hbase configuration hbase-site.xml command line

GeKas — Wed, 18 Jul 2018 08:22:17 GMT

You can try using the API

https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_intro_api.html

https://cloudera.github.io/cm_api/apidocs/v19/index.html

Re: Hbase configuration hbase-site.xml command line

yassine24 — Wed, 18 Jul 2018 11:30:20 GMT

I already used the API.

My problem is that I generated a hbase-site.xml with new value for severals parameters.

Now I want to import it into Cloudera Manager with API.

So I am looking for a way to put the xml file into Cloudera Manager without using the Cloudera manager UI.

With the Api I succed to retrieve the attributes : "hbase_service_config_safety_valve". But I don't find how can I modify it in my python code.

Is there a way to put some xml code directly into Cloudera Manager without GUI ?

Thanks

Re: Hbase configuration hbase-site.xml command line

GeKas — Wed, 18 Jul 2018 11:49:36 GMT

From command line I don't know if there is a way to upload xml for a specific service. You can do it for the cluster.

But you should check

https://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_services_-serviceName-_config.html

and

https://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_services_-serviceName-_roles_-roleName-_config.html

You can also modify parameters by using PUT instead of GET

Re: Hbase configuration hbase-site.xml command line

yassine24 — Wed, 18 Jul 2018 12:45:18 GMT

Can you explain me what I have to use for upload xml for the cluster ?

Thank you

Re: Hbase configuration hbase-site.xml command line

GeKas — Wed, 18 Jul 2018 13:25:42 GMT

My mistake. Cloudera works with json not xml.

Anyhow, to upload complete cluster config you can do it by following https://gist.github.com/gwenshap/7044525 but this is for serious cases.

On the other hand, you can push only HBase configuration with the previous links I send you. You just have to prepare config in json format.

Re: Hbase configuration hbase-site.xml command line

bgooley — Wed, 18 Jul 2018 21:40:16 GMT

@yassine24,

This shows how to update a service configuration

http://cloudera.github.io/cm_api/docs/python-client/#configuring-services-and-roles

You need to update the config with the attribute and value.

The configuration is JSON format, but the safety valve you want is in XML format.

An example of how to update a safety valve (hdfs in this case) via REST API is here:

curl -iv -X PUT -H "Content-Type:application/json" -H "Accept:application/json" -d '{"items":[{ "name": "core_site_safety_valve","value": "<property><name>hadoop.proxyuser.ztsps.users</name><value>*</value></property><property><name>hadoop.proxyuser.ztsps.groups</name><value>*</value></property>"}]}' http://admin:admin@10.1.0.1:7180/api/v12/clusters/cluster/services/hdfs/config

I am pretty sure you can pass the JSON as shown above in the -d argument hbase.update_config() or whatever