Member since
01-08-2020
3
Posts
0
Kudos Received
0
Solutions
09-24-2021
12:48 PM
For reference, I resolve exactly the same issue by starting ambari-agent on the Ambari server node:
# ambari-agent start
IIRC, ambari-agent is not required to be running to enable Kerberos with Ambari 2.7.4. Don't know why it's required on 2.7.5.
------
Some further DEBUG info in ambari-server.log:
1. the keytabs were generated on ambari-server tmp directory /var/lib/ambari-server/data/tmp/ but failed to copy to /etc/security/keytabs/ on ambari-agent nodes:
2021-09-22 11:02:10,443 DEBUG [process-identity-task-110-thread-0] KerberosKeytabDAO:111 - Loading keytabs by principal name and host took 0ms
2021-09-22 11:02:10,443 INFO [process-identity-task-110-thread-0] CreateKeytabFilesServerAction:198 - Creating keytab file for hdp31-092221@EXAMPLE.COM on host node2.example.com
2021-09-22 11:02:10,444 DEBUG [process-identity-task-110-thread-0] CreateKeytabFilesServerAction:325 - Creating keytab for hdp31-092221@EXAMPLE.COM with kvno 0
2021-09-22 11:02:10,444 INFO [process-identity-task-110-thread-0] CreateKeytabFilesServerAction:257 - Successfully created keytab file for hdp31-092221@EXAMPLE.COM at /var/lib/ambari-server/data/tmp/.ambari_1632322925100-0.d/node2.example.com/6988997fbb62486a193d1de07e235fd23ffad1eb2b68837a6167409156b44444
2. "Could not inject keytab into command" error then encountered due to "Missing keytabs" on ambari-agent nodes:
......
2021-09-22 11:02:11,406 INFO [ambari-action-scheduler] AgentCommandsPublisher:124 - AgentCommandsPublisher.sendCommands: sending ExecutionCommand for host node1.example.com, role KERBEROS_CLIENT, roleCommand CUSTOM_COMMAND, and command ID 6-4, task ID 111
2021-09-22 11:02:11,406 INFO [ambari-action-scheduler] AgentCommandsPublisher:130 - SET_KEYTAB called
2021-09-22 11:02:11,412 DEBUG [ambari-action-scheduler] ActionScheduler:575 - Scheduler finished work.
2021-09-22 11:02:11,413 WARN [ambari-action-scheduler] ActionScheduler:353 - Exception received
org.apache.ambari.server.AmbariException: Could not inject keytab into command
at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.populateExecutionCommandsClusters(AgentCommandsPublisher.java:134)
at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.sendAgentCommand(AgentCommandsPublisher.java:92)
at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:557)
at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:347)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.ambari.server.AmbariException: Could not inject keytabs to enable kerberos
at org.apache.ambari.server.events.publishers.AgentCommandsPublisher$KerberosCommandParameterProcessor.process(AgentCommandsPublisher.java:261)
at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.injectKeytab(AgentCommandsPublisher.java:184)
at org.apache.ambari.server.events.publishers.AgentCommandsPublisher.populateExecutionCommandsClusters(AgentCommandsPublisher.java:132)
... 4 more
... View more