Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

how to fetch service level configuration using REST API

avatar
Expert Contributor

Can someone help me to fetch service level config using ambari REST API?

For example, we would like to have the following config value pulled using ambari REST API calls.

hive.server2.transport.mode

hive.txn.timeout

Thanks,

1 ACCEPTED SOLUTION

avatar
Expert Contributor

I would take a look at the Apache Ambari API reference.

I think that this link might be of use: https://github.com/apache/ambari/blob/trunk/ambari...

For a list of all configurations to the API you could make a call similar to the following

GET /api/v1/clusters/cluster/configurations

Then if you wanted to get all of the configuration versions for Hive you could make the call to

GET /api/v1/clusters/cluster/configurations?type=hive-site

And from that list if you picked one of the versions you would make a call like:

/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723

The response would look something like:

{
  "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
  "items" : [
    {
      "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
      "tag" : "version1447446748723",
      "type" : "hive-site",
      "version" : 4,
      "Config" : {
        "cluster_name" : "Sandbox",
        "stack_id" : "HDP-2.3"
      },
      "properties" : {
        "ambari.hive.db.schema.name" : "hive",
        "atlas.cluster.name" : "Sandbox",
        "atlas.hook.hive.synchronous" : "true",
        "atlas.rest.address" : "http://sandbox.hortonworks.com:21000",
        "datanucleus.autoCreateSchema" : "false",
        "datanucleus.cache.level2.type" : "none",
        "hive.auto.convert.join" : "true",
        "hive.auto.convert.join.noconditionaltask" : "true",
        "hive.auto.convert.join.noconditionaltask.size" : "357913941",
        "hive.auto.convert.sortmerge.join" : "true",
        "hive.auto.convert.sortmerge.join.to.mapjoin" : "false",
        "hive.cbo.enable" : "true",
        "hive.cli.print.header" : "false",
        "hive.cluster.delegation.token.store.class" : "org.apache.hadoop.hive.thrift.ZooKeeperTokenStore",
        "hive.cluster.delegation.token.store.zookeeper.connectString" : "sandbox.hortonworks.com:2181",
        "hive.cluster.delegation.token.store.zookeeper.znode" : "/hive/cluster/delegation",
        "hive.compactor.abortedtxn.threshold" : "1000",
        "hive.compactor.check.interval" : "300s",
        "hive.compactor.delta.num.threshold" : "10",
        "hive.compactor.delta.pct.threshold" : "0.1f", 


  "hive.security.authorization.enabled" : "true",
        "hive.security.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory",
        "hive.security.metastore.authenticator.manager" : "org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator",
        "hive.security.metastore.authorization.auth.reads" : "true",
        "hive.security.metastore.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider",
        "hive.server2.allow.user.substitution" : "true",
        "hive.server2.authentication" : "NONE",
        "hive.server2.authentication.spnego.keytab" : "HTTP/_HOST@EXAMPLE.COM",
        "hive.server2.authentication.spnego.principal" : "/etc/security/keytabs/spnego.service.keytab",
        "hive.server2.enable.doAs" : "false",
        "hive.server2.logging.operation.enabled" : "true",
        "hive.server2.logging.operation.log.location" : "${system:java.io.tmpdir}/${system:user.name}/operation_logs",
        "hive.server2.support.dynamic.service.discovery" : "true",
        "hive.server2.table.type.mapping" : "CLASSIC",
        "hive.server2.tez.default.queues" : "default",
        "hive.server2.tez.initialize.default.sessions" : "false",
        "hive.server2.tez.sessions.per.default.queue" : "1",
        "hive.server2.thrift.http.path" : "cliservice",
        "hive.server2.thrift.http.port" : "10001",
        "hive.server2.thrift.max.worker.threads" : "500",
        "hive.server2.thrift.port" : "10000",
        "hive.server2.thrift.sasl.qop" : "auth",
        "hive.server2.transport.mode" : "binary",
        "hive.server2.use.SSL" : "false",
        "hive.server2.zookeeper.namespace" : "hiveserver2",
        "hive.smbjoin.cache.rows" : "10000",
        "hive.stats.autogather" : "true",
        "hive.stats.dbclass" : "fs",

From this you can retrieve the correct property and it's value.

View solution in original post

2 REPLIES 2

avatar
Expert Contributor

I would take a look at the Apache Ambari API reference.

I think that this link might be of use: https://github.com/apache/ambari/blob/trunk/ambari...

For a list of all configurations to the API you could make a call similar to the following

GET /api/v1/clusters/cluster/configurations

Then if you wanted to get all of the configuration versions for Hive you could make the call to

GET /api/v1/clusters/cluster/configurations?type=hive-site

And from that list if you picked one of the versions you would make a call like:

/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723

The response would look something like:

{
  "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
  "items" : [
    {
      "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
      "tag" : "version1447446748723",
      "type" : "hive-site",
      "version" : 4,
      "Config" : {
        "cluster_name" : "Sandbox",
        "stack_id" : "HDP-2.3"
      },
      "properties" : {
        "ambari.hive.db.schema.name" : "hive",
        "atlas.cluster.name" : "Sandbox",
        "atlas.hook.hive.synchronous" : "true",
        "atlas.rest.address" : "http://sandbox.hortonworks.com:21000",
        "datanucleus.autoCreateSchema" : "false",
        "datanucleus.cache.level2.type" : "none",
        "hive.auto.convert.join" : "true",
        "hive.auto.convert.join.noconditionaltask" : "true",
        "hive.auto.convert.join.noconditionaltask.size" : "357913941",
        "hive.auto.convert.sortmerge.join" : "true",
        "hive.auto.convert.sortmerge.join.to.mapjoin" : "false",
        "hive.cbo.enable" : "true",
        "hive.cli.print.header" : "false",
        "hive.cluster.delegation.token.store.class" : "org.apache.hadoop.hive.thrift.ZooKeeperTokenStore",
        "hive.cluster.delegation.token.store.zookeeper.connectString" : "sandbox.hortonworks.com:2181",
        "hive.cluster.delegation.token.store.zookeeper.znode" : "/hive/cluster/delegation",
        "hive.compactor.abortedtxn.threshold" : "1000",
        "hive.compactor.check.interval" : "300s",
        "hive.compactor.delta.num.threshold" : "10",
        "hive.compactor.delta.pct.threshold" : "0.1f", 


  "hive.security.authorization.enabled" : "true",
        "hive.security.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory",
        "hive.security.metastore.authenticator.manager" : "org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator",
        "hive.security.metastore.authorization.auth.reads" : "true",
        "hive.security.metastore.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider",
        "hive.server2.allow.user.substitution" : "true",
        "hive.server2.authentication" : "NONE",
        "hive.server2.authentication.spnego.keytab" : "HTTP/_HOST@EXAMPLE.COM",
        "hive.server2.authentication.spnego.principal" : "/etc/security/keytabs/spnego.service.keytab",
        "hive.server2.enable.doAs" : "false",
        "hive.server2.logging.operation.enabled" : "true",
        "hive.server2.logging.operation.log.location" : "${system:java.io.tmpdir}/${system:user.name}/operation_logs",
        "hive.server2.support.dynamic.service.discovery" : "true",
        "hive.server2.table.type.mapping" : "CLASSIC",
        "hive.server2.tez.default.queues" : "default",
        "hive.server2.tez.initialize.default.sessions" : "false",
        "hive.server2.tez.sessions.per.default.queue" : "1",
        "hive.server2.thrift.http.path" : "cliservice",
        "hive.server2.thrift.http.port" : "10001",
        "hive.server2.thrift.max.worker.threads" : "500",
        "hive.server2.thrift.port" : "10000",
        "hive.server2.thrift.sasl.qop" : "auth",
        "hive.server2.transport.mode" : "binary",
        "hive.server2.use.SSL" : "false",
        "hive.server2.zookeeper.namespace" : "hiveserver2",
        "hive.smbjoin.cache.rows" : "10000",
        "hive.stats.autogather" : "true",
        "hive.stats.dbclass" : "fs",

From this you can retrieve the correct property and it's value.

avatar
Expert Contributor

Thanks very much. This is exactly what i was looking for.