what's the issue . Can you please explain at bit more . ... View more

Thanks allot @Shelton , it worked and i'm able to see expected results.  ... View more

Hello Shelton,    Thanks allot, I followed all the steps and was able to enable Kerberos in ambari GUI , but i want to know how to check that's enabled and i tried to perform few commands in Datanode (hadoop services installed) and that the error i got .    hdfs dfs ls /           Please help me how to verify that cluster is kerberized and how to generate tickets and see the authentication . Thanks allot Shelton. @Shelton  ... View more

      vim /etc/hosts   # x.x.x.x   localhost localhost.localdomain localhost4 localhost4.localdomain4 #::1         localhost localhost.localdomain localhost6 localhost6.localdomain6   x.x.x.x   FQDN     server x.x.x.x   ESXI-host1 x.x.x.x   ESXI-host2 x.x.x.x   ESXI-host3   vim /var/kerberos/krb5kdc/kadm5.acl */admin@DOMAIN.COM *   vim /var/kerberos/krb5kdc/kdc.conf [kdcdefaults]  kdc_ports = 88  kdc_tcp_ports = 88 [realms]  DOMAIN.COM= { #master_key_type = aes256-cts  acl_file = /var/kerberos/krb5kdc/kadm5.acl  dict_file = /usr/share/dict/words  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal  } [realms]  DOMAIN.COM= {  master_key_type = des-cbc-crc  database_name = /var/kerberos/krb5kdc/principal  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab  supported_enctypes = des-cbc-crc:normal des3-cbc-raw:normal des3-cbc-sha1:normal des-cbc-crc:v4 des-cbc-crc:afs3  kadmind_port = 749  acl_file = /var/kerberos/krb5kdc/kadm5.acl  dict_file = /usr/dict/words     list _ Principlas [root@KERBEROSSERVER~]# kadmin.local Authenticating as principal root/admin@DOMAIN.COM with password. kadmin.local:  list_principals HTTP/datanodeFQDN@DOMAIN.COM HTTP/np-dev1-hdp315-namenode-01.DOMAIN.COM@DOMAIN.COM K/M@DOMAIN.COM activity_analyzer/datanodeFQDN@DOMAIN.COM activity_explorer/datanodeFQDN@DOMAIN.COM admin/admin@DOMAIN.COM ambari-qa-hdpcluster@DOMAIN.COM ambari-server-hdpcluster@DOMAIN.COM amshbase/datanodeFQDN@DOMAIN.COM amsmon/datanodeFQDN@DOMAIN.COM amszk/datanodeFQDN@DOMAIN.COM atlas/datanodeFQDN@DOMAIN.COM dn/datanodeFQDN@DOMAIN.COM hbase-hdpcluster@DOMAIN.COM hbase/datanodeFQDN@DOMAIN.COM hdfs-hdpcluster@DOMAIN.COM hdpcluster-072221@DOMAIN.COM hdpcluster-072321@DOMAIN.COM hive/datanodeFQDN@DOMAIN.COM infra-solr/datanodeFQDN@DOMAIN.COM jhs/datanodeFQDN@DOMAIN.COM kadmin/admin@DOMAIN.COM kadmin/changepw@DOMAIN.COM kadmin/KERBEROSSERVERDOMAIN.COM@DOMAIN.COM kafka/datanodeFQDN@DOMAIN.COM kiprop/KERBEROSSERVERDOMAIN.COM@DOMAIN.COM krbtgt/DOMAIN.COM@DOMAIN.COM nm/datanodeFQDN@DOMAIN.COM nn/datanodeFQDN@DOMAIN.COM rm/datanodeFQDN@DOMAIN.COM root/admin@DOMAIN.COM spark-hdpcluster@DOMAIN.COM spark/datanodeFQDN@DOMAIN.COM spark_atlas@DOMAIN.COM yarn-ats-hbase/datanodeFQDN@DOMAIN.COM yarn-ats-hdpcluster@DOMAIN.COM yarn/datanodeFQDN@DOMAIN.COM zeppelin-hdpcluster@DOMAIN.COM zookeeper/datanodeFQDN@DOMAIN.COMkadmin.local:   AMBARILOGS   m   2021-07-23 12:43:42,666 WARN [Stack Version Loading Thread] RepoVdfCallable:142 - Pastebin.com     2021-07-23 12:43:50,595 WARN [main] Errors:173 - The following warnings hav - Pastebin.com           ... View more

(I setup kerberos in different server , not on namenode or datanode !! ) Is this a good practice!!!      After changing the krb5.config and kdbc.conf , tried to enable kerberos in ambari GUI by providing admin credentials and stuck at same place .    logs from ambari-server :  2021-07-23 02:15:18,218 WARN [ambari-action-scheduler] ActionScheduler:353 - Exception received org.apache.ambari.server.AmbariException: Could not inject keytab into command at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.populateExecutionCommandsClusters(AgentCommandsPublisher.java:134) at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.sendAgentCommand(AgentCommandsPublisher.java:92) at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:557) at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:347) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.ambari.server.AmbariException: Could not inject keytabs to enable kerberos at org.apache.ambari.server.events.publishers.AgentCommandsPublisher$KerberosCommandParameterProcessor.process(AgentCommandsPublisher.java:261) at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.injectKeytab(AgentCommandsPublisher.java:184) at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.populateExecutionCommandsClusters(AgentCommandsPublisher.java:132) ... 4 more 2021-07-23 02:15:31,409 WARN [ambari-client-thread-194] Errors:173 - The following warnings have been detected with resource and/or provider classes: WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.TaskService.getComponents(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity. WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.TaskService.getTask(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity. 2021-07-23 02:15:31,409 WARN [ambari-client-thread-194] Errors:173 - The following warnings have been detected with resource and/or provider classes: WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.TaskService.getComponents(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo), should not consume any entity. WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.ambari.server.api.services.TaskService.getTask(java.lang.String,javax.ws.rs.core.HttpHeaders,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity. 2021-07-23 02:15:34,388 INFO [Thread-20] AbstractPoolBackedDataSource:212 - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 2wkjnfai1m9lw9qg7p0r4|1f15d346, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.postgresql.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 2wkjnfai1m9lw9qg7p0r4|1f15d346, idleConnectionTestPeriod -> 50, initialPoolSize -> 3, jdbcUrl -> jdbc:postgresql://localhost/ambari, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 5, maxStatements -> 0, maxStatementsPerConnection -> 120, minPoolSize -> 1, numHelperThreads -> 3, preferredTestQuery -> select 0, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> true, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ] 2021-07-23 02:15:34,452 INFO [Thread-20] JobStoreTX:866 - Freed 0 triggers from 'acquired' / 'blocked' state. 2021-07-23 02:15:34,462 INFO [Thread-20] JobStoreTX:876 - Recovering 0 jobs that were in-progress at the time of the last shut-down. 2021-07-23 02:15:34,462 INFO [Thread-20] JobStoreTX:889 - Recovery complete. 2021-07-23 02:15:34,463 INFO [Thread-20] JobStoreTX:896 - Removed 0 'complete' triggers. 2021-07-23 02:15:34,464 INFO [Thread-20] JobStoreTX:901 - Removed 0 stale fired job entries. 2021-07-23 02:15:34,465 INFO [Thread-20] QuartzScheduler:547 - Scheduler ExecutionScheduler_$_NON_CLUSTERED started.   NAMENODE :      DATANODE:    kadmin.local:  list_principals  HTTP/datanode.example.com@EXAMPLE.COM HTTP/namenode.example.com@EXAMPLE.COM K/M@EXAMPLE.COM activity_analyzer/datanode.example.com@EXAMPLE.COM activity_explorer/datanode.example.com@EXAMPLE.COM admin/admin@EXAMPLE.COM ambari-qa-hdpcluster@EXAMPLE.COM ambari-server-hdpcluster@EXAMPLE.COM amshbase/datanode.example.com@EXAMPLE.COM amsmon/datanode.example.com@EXAMPLE.COM amszk/datanode.example.com@EXAMPLE.COM atlas/datanode.example.com@EXAMPLE.COM dn/datanode.example.com@EXAMPLE.COM hbase-hdpcluster@EXAMPLE.COM hbase/datanode.example.com@EXAMPLE.COM hdfs-hdpcluster@EXAMPLE.COM hdpcluster-072221@EXAMPLE.COM hdpcluster-072321@EXAMPLE.COM hive/datanode.example.com@EXAMPLE.COM infra-solr/datanode.example.com@EXAMPLE.COM jhs/datanode.example.com@EXAMPLE.COM kadmin/admin@EXAMPLE.COM kadmin/changepw@EXAMPLE.COM kadmin/np-devops-inventory.example.com@EXAMPLE.COM kafka/datanode.example.com@EXAMPLE.COM kiprop/np-devops-inventory.example.com@EXAMPLE.COM krbtgt/EXAMPLE.COM@EXAMPLE.COM nm/datanode.example.com@EXAMPLE.COM nn/datanode.example.com@EXAMPLE.COM rm/datanode.example.com@EXAMPLE.COM root/admin@EXAMPLE.COM spark-hdpcluster@EXAMPLE.COM spark/datanode.example.com@EXAMPLE.COM spark_atlas@EXAMPLE.COM yarn-ats-hbase/datanode.example.com@EXAMPLE.COM yarn-ats-hdpcluster@EXAMPLE.COM yarn/datanode.example.com@EXAMPLE.COM zookeeper/datanode.example.com@EXAMPLE.COM ... View more