Obtaining a listing of expected Kerberos Identities from Ambari

[#1]

Once Kerberos is enabled, it is possible to get a listing of the expected Kerberos principals and keytab files. This data is typically used when manually managing the Kerberos Identities for the Ambari cluster and may be easily downloaded as a CSV file from the Enable Kerberos Wizard in the Ambari UI.

After Kerberos is enabled in Ambari, details about the expected Kerberos Identities may be obtained using the following REST API call:

GET /api/v1/clusters/:cluster_name/kerberos_identities

Note: replace :cluster_name with the name of your cluster.

The result of this query is a JSON formatted document containing a high level listing of the expected identities.

{
  "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities",
  "items" : [
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost-1.example.com%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "host_name" : "host-1.example.com",
        "principal_name" : "HTTP/host-1.example.com@EXAMPLE.COM"
      }
    },
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa-c1%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "host_name" : "host-1.example.com",
        "principal_name" : "ambari-qa-c1@EXAMPLE.COM"
      }
    },
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/dn%2Fhost-1.example.com%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "host_name" : "host-1.example.com",
        "principal_name" : "dn/host-1.example.com@EXAMPLE.COM"
      }
    },
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/hdfs-c1%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "host_name" : "host-1.example.com",
        "principal_name" : "hdfs-c1@EXAMPLE.COM"
      }
    },
...

To get more information on each identity follow the provided URLs in the output. For example, for ambari-qa-c1@EXAMPLE.COM the URL would be:

http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa-c1%40EXAMPLE.COM

Which will yield something like:

[
  {
    "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa-c1%40EXAMPLE.COM",
    "KerberosIdentity" : {
      "cluster_name" : "c1",
      "description" : "/smokeuser",
      "host_name" : "host-1.example.com",
      "keytab_file_group" : "hadoop",
      "keytab_file_group_access" : "r",
      "keytab_file_installed" : "true",
      "keytab_file_mode" : "440",
      "keytab_file_owner" : "ambari-qa",
      "keytab_file_owner_access" : "r",
      "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
      "principal_local_username" : "ambari-qa",
      "principal_name" : "ambari-qa-c1@EXAMPLE.COM",
      "principal_type" : "USER"
    }
  },
  {
    "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa-c1%40EXAMPLE.COM",
    "KerberosIdentity" : {
      "cluster_name" : "c1",
      "description" : "/smokeuser",
      "host_name" : "host-2.example.com",
      "keytab_file_group" : "hadoop",
      "keytab_file_group_access" : "r",
      "keytab_file_installed" : "true",
      "keytab_file_mode" : "440",
      "keytab_file_owner" : "ambari-qa",
      "keytab_file_owner_access" : "r",
      "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
      "principal_local_username" : "ambari-qa",
      "principal_name" : "ambari-qa-c1@EXAMPLE.COM",
      "principal_type" : "USER"
    }
  },
...

To get the all of the data in one query, indicate that you want all of the field data by appending fields=* to the original query:

GET /api/v1/clusters/:cluster_name/kerberos_identities?fields=*

This will yield something like the following:

{
  "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
  "items" : [
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost-1.example.com%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "description" : "/spnego",
        "host_name" : "host-1.example.com",
        "keytab_file_group" : "hadoop",
        "keytab_file_group_access" : "r",
        "keytab_file_installed" : "true",
        "keytab_file_mode" : "440",
        "keytab_file_owner" : "root",
        "keytab_file_owner_access" : "r",
        "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
        "principal_local_username" : null,
        "principal_name" : "HTTP/host-1.example.com@EXAMPLE.COM",
        "principal_type" : "SERVICE"
      }
    },
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa-c1%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "description" : "/smokeuser",
        "host_name" : "host-1.example.com",
        "keytab_file_group" : "hadoop",
        "keytab_file_group_access" : "r",
        "keytab_file_installed" : "true",
        "keytab_file_mode" : "440",
        "keytab_file_owner" : "ambari-qa",
        "keytab_file_owner_access" : "r",
        "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
        "principal_local_username" : "ambari-qa",
        "principal_name" : "ambari-qa-c1@EXAMPLE.COM",
        "principal_type" : "USER"
      }
    },
    {
      "href" : "http://ambari-server:8080/api/v1/clusters/c1/kerberos_identities/dn%2Fhost-1.example.com%40EXAMPLE.COM",
      "KerberosIdentity" : {
        "cluster_name" : "c1",
        "description" : "datanode_dn",
        "host_name" : "host-1.example.com",
        "keytab_file_group" : "hadoop",
        "keytab_file_group_access" : "",
        "keytab_file_installed" : "true",
        "keytab_file_mode" : "400",
        "keytab_file_owner" : "hdfs",
        "keytab_file_owner_access" : "r",
        "keytab_file_path" : "/etc/security/keytabs/dn.service.keytab",
        "principal_local_username" : "hdfs",
        "principal_name" : "dn/host-1.example.com@EXAMPLE.COM",
        "principal_type" : "SERVICE"
      }
    },
...

In many cases, this may be good enough. However there may be a need to get this data in a different format, especially when using the data in a script. In that case, it is possible to retrieve this as CSV formatted data by appending format=CSV to the query:

GET /api/v1/clusters/:cluster_name/kerberos_identities?fields=*&format=CSV

The CSV formatted data will look something like:

host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
host-1.example.com,/spnego,HTTP/host-1.example.com@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host-1.example.com,/smokeuser,ambari-qa-c1@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host-1.example.com,datanode_dn,dn/host-1.example.com@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
...
host-1.example.com,hdfs,hdfs-c1@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host-1.example.com,namenode_nn,nn/host-1.example.com@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host-1.example.com,zookeeper_zk,zookeeper/host-1.example.com@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
host-2.example.com,/spnego,HTTP/host-2.example.com@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host-2.example.com,/smokeuser,ambari-qa-c1@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host-2.example.com,datanode_dn,dn/host-2.example.com@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host-2.example.com,secondary_namenode_nn,nn/host-2.example.com@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host-2.example.com,zookeeper_zk,zookeeper/host-2.example.com@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true

[#2]

Thank you so much!

This helps greatly.

Quick question: when adding a new host to the cluster, until now, I needed to manually create the services in the KDC, and generate the keytabs on the host. But as of now, thanks to the kerberos_identities API call, does it mean I can just track the changes in kerberos_identities to automatically detect the identities related to the new host, and script the services and keytabs creation? Another way to put it: how soon, in the 'adding a new host' process, are the kerberos identities updated with the new identities?

[#3]

I am glad you found this article useful.

I do not really understand your question, so I hope this helps to clarify things.

As of Ambari 2.0, Ambari will automatically create Kerberos identities (principal name and keytab files) for newly added components - whether they are added to existing or new hosts. The new principals and keytab files will be available to the relevant component(s) before they are started or rather just after they are installed. This is, as long as Ambari it setup to manage the Kerberos identities for the cluster - which can be optionally turned off as of some version later and Ambari 2.0.

The "Kerberos Identities" API call, does not really come into play when Ambari is managing the cluster's Kerberos identities. However, it can be used for informational purposes for you. When Ambari is not managing the cluster's Kerberos identities, the "Kerberos Identities" API call is useful as it will give you a listing of the principal name and keytab files it expects to exist. You can use this information to manually create the needed data. Unfortunately, though, the returned data does not properly indicate which identity records are _new_. So it is up to the consumer to figure this out.

[#4]

Hi Robert Levas,

Thanks for these help. I used to download kerberos.csv file in previous version but now i'm facing issue.

I used to obtain the Kerberos.csv file using command " GET /api/v1/clusters/:cluster_name/kerberos_identities?fields=*&format=CSV " as mentioned by you in previous version of HDP i.e. HDP 2.4, But I'm not able to download it with HDP 2.5.3 and Amabri server 2.2. Is there any change in path of kerberos_identities.

I have attached the screenshot where one can see that an empty file has downloaded with no Principals and keytabs.

[#5]

Hi @Mahesh Thumar

Given the same version of Ambari (version 2.2), I am not sure why the stack would make a difference. On that note, as far as I know, nothing has changed for that entry point since Ambari 2.1.0. When I get a chance I will see if I can figure out what the deal is.

[#6]

Hi Robert,

I am working along with Mahesh regarding this issue ,one correction with respect to his question ,we are using ambari version 2.4 and not 2.2

Regards

Chandan Singh

[#7]

I just tried this and had no issues.

• Ambari 2.4.2/HDP 2.4
  • curl command worked fine
• Express upgrade to HDP 2.5 (Ambari 2.4.2/HDP 2.5)
  • curl command worked fine

Check your ambari.log file to see if there are any interesting errors.