Support Questions

Find answers, ask questions, and share your expertise

how to fetch service level configuration using REST API

avatar
Expert Contributor

Can someone help me to fetch service level config using ambari REST API?

For example, we would like to have the following config value pulled using ambari REST API calls.

hive.server2.transport.mode

hive.txn.timeout

Thanks,

1 ACCEPTED SOLUTION

avatar
Expert Contributor

I would take a look at the Apache Ambari API reference.

I think that this link might be of use: https://github.com/apache/ambari/blob/trunk/ambari...

For a list of all configurations to the API you could make a call similar to the following

GET /api/v1/clusters/cluster/configurations

Then if you wanted to get all of the configuration versions for Hive you could make the call to

GET /api/v1/clusters/cluster/configurations?type=hive-site

And from that list if you picked one of the versions you would make a call like:

/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723

The response would look something like:

{
  "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
  "items" : [
    {
      "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
      "tag" : "version1447446748723",
      "type" : "hive-site",
      "version" : 4,
      "Config" : {
        "cluster_name" : "Sandbox",
        "stack_id" : "HDP-2.3"
      },
      "properties" : {
        "ambari.hive.db.schema.name" : "hive",
        "atlas.cluster.name" : "Sandbox",
        "atlas.hook.hive.synchronous" : "true",
        "atlas.rest.address" : "http://sandbox.hortonworks.com:21000",
        "datanucleus.autoCreateSchema" : "false",
        "datanucleus.cache.level2.type" : "none",
        "hive.auto.convert.join" : "true",
        "hive.auto.convert.join.noconditionaltask" : "true",
        "hive.auto.convert.join.noconditionaltask.size" : "357913941",
        "hive.auto.convert.sortmerge.join" : "true",
        "hive.auto.convert.sortmerge.join.to.mapjoin" : "false",
        "hive.cbo.enable" : "true",
        "hive.cli.print.header" : "false",
        "hive.cluster.delegation.token.store.class" : "org.apache.hadoop.hive.thrift.ZooKeeperTokenStore",
        "hive.cluster.delegation.token.store.zookeeper.connectString" : "sandbox.hortonworks.com:2181",
        "hive.cluster.delegation.token.store.zookeeper.znode" : "/hive/cluster/delegation",
        "hive.compactor.abortedtxn.threshold" : "1000",
        "hive.compactor.check.interval" : "300s",
        "hive.compactor.delta.num.threshold" : "10",
        "hive.compactor.delta.pct.threshold" : "0.1f", 


  "hive.security.authorization.enabled" : "true",
        "hive.security.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory",
        "hive.security.metastore.authenticator.manager" : "org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator",
        "hive.security.metastore.authorization.auth.reads" : "true",
        "hive.security.metastore.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider",
        "hive.server2.allow.user.substitution" : "true",
        "hive.server2.authentication" : "NONE",
        "hive.server2.authentication.spnego.keytab" : "HTTP/_HOST@EXAMPLE.COM",
        "hive.server2.authentication.spnego.principal" : "/etc/security/keytabs/spnego.service.keytab",
        "hive.server2.enable.doAs" : "false",
        "hive.server2.logging.operation.enabled" : "true",
        "hive.server2.logging.operation.log.location" : "${system:java.io.tmpdir}/${system:user.name}/operation_logs",
        "hive.server2.support.dynamic.service.discovery" : "true",
        "hive.server2.table.type.mapping" : "CLASSIC",
        "hive.server2.tez.default.queues" : "default",
        "hive.server2.tez.initialize.default.sessions" : "false",
        "hive.server2.tez.sessions.per.default.queue" : "1",
        "hive.server2.thrift.http.path" : "cliservice",
        "hive.server2.thrift.http.port" : "10001",
        "hive.server2.thrift.max.worker.threads" : "500",
        "hive.server2.thrift.port" : "10000",
        "hive.server2.thrift.sasl.qop" : "auth",
        "hive.server2.transport.mode" : "binary",
        "hive.server2.use.SSL" : "false",
        "hive.server2.zookeeper.namespace" : "hiveserver2",
        "hive.smbjoin.cache.rows" : "10000",
        "hive.stats.autogather" : "true",
        "hive.stats.dbclass" : "fs",

From this you can retrieve the correct property and it's value.

View solution in original post

2 REPLIES 2

avatar
Expert Contributor

I would take a look at the Apache Ambari API reference.

I think that this link might be of use: https://github.com/apache/ambari/blob/trunk/ambari...

For a list of all configurations to the API you could make a call similar to the following

GET /api/v1/clusters/cluster/configurations

Then if you wanted to get all of the configuration versions for Hive you could make the call to

GET /api/v1/clusters/cluster/configurations?type=hive-site

And from that list if you picked one of the versions you would make a call like:

/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723

The response would look something like:

{
  "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
  "items" : [
    {
      "href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=hive-site&tag=version1447446748723",
      "tag" : "version1447446748723",
      "type" : "hive-site",
      "version" : 4,
      "Config" : {
        "cluster_name" : "Sandbox",
        "stack_id" : "HDP-2.3"
      },
      "properties" : {
        "ambari.hive.db.schema.name" : "hive",
        "atlas.cluster.name" : "Sandbox",
        "atlas.hook.hive.synchronous" : "true",
        "atlas.rest.address" : "http://sandbox.hortonworks.com:21000",
        "datanucleus.autoCreateSchema" : "false",
        "datanucleus.cache.level2.type" : "none",
        "hive.auto.convert.join" : "true",
        "hive.auto.convert.join.noconditionaltask" : "true",
        "hive.auto.convert.join.noconditionaltask.size" : "357913941",
        "hive.auto.convert.sortmerge.join" : "true",
        "hive.auto.convert.sortmerge.join.to.mapjoin" : "false",
        "hive.cbo.enable" : "true",
        "hive.cli.print.header" : "false",
        "hive.cluster.delegation.token.store.class" : "org.apache.hadoop.hive.thrift.ZooKeeperTokenStore",
        "hive.cluster.delegation.token.store.zookeeper.connectString" : "sandbox.hortonworks.com:2181",
        "hive.cluster.delegation.token.store.zookeeper.znode" : "/hive/cluster/delegation",
        "hive.compactor.abortedtxn.threshold" : "1000",
        "hive.compactor.check.interval" : "300s",
        "hive.compactor.delta.num.threshold" : "10",
        "hive.compactor.delta.pct.threshold" : "0.1f", 


  "hive.security.authorization.enabled" : "true",
        "hive.security.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory",
        "hive.security.metastore.authenticator.manager" : "org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator",
        "hive.security.metastore.authorization.auth.reads" : "true",
        "hive.security.metastore.authorization.manager" : "org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider",
        "hive.server2.allow.user.substitution" : "true",
        "hive.server2.authentication" : "NONE",
        "hive.server2.authentication.spnego.keytab" : "HTTP/_HOST@EXAMPLE.COM",
        "hive.server2.authentication.spnego.principal" : "/etc/security/keytabs/spnego.service.keytab",
        "hive.server2.enable.doAs" : "false",
        "hive.server2.logging.operation.enabled" : "true",
        "hive.server2.logging.operation.log.location" : "${system:java.io.tmpdir}/${system:user.name}/operation_logs",
        "hive.server2.support.dynamic.service.discovery" : "true",
        "hive.server2.table.type.mapping" : "CLASSIC",
        "hive.server2.tez.default.queues" : "default",
        "hive.server2.tez.initialize.default.sessions" : "false",
        "hive.server2.tez.sessions.per.default.queue" : "1",
        "hive.server2.thrift.http.path" : "cliservice",
        "hive.server2.thrift.http.port" : "10001",
        "hive.server2.thrift.max.worker.threads" : "500",
        "hive.server2.thrift.port" : "10000",
        "hive.server2.thrift.sasl.qop" : "auth",
        "hive.server2.transport.mode" : "binary",
        "hive.server2.use.SSL" : "false",
        "hive.server2.zookeeper.namespace" : "hiveserver2",
        "hive.smbjoin.cache.rows" : "10000",
        "hive.stats.autogather" : "true",
        "hive.stats.dbclass" : "fs",

From this you can retrieve the correct property and it's value.

avatar
Expert Contributor

Thanks very much. This is exactly what i was looking for.