I tried to install kerberos in the hadoop cluster. i see the following error on all the hosts. On host eursrcux999 role KERBEROS_CLIENT in invalid state. Invalid transition. Invalid event: HOST_SVCCOMP_OP_IN_PROGRESS at INSTALL_FAILED

[#1]

@Neeraj Sabharwal

[#2]

@rajeshbabu chintaguntla

[#3]

@sunil kanthety - looks like some issues with the installation of KERBEROS_CLIENT. can you please remove it using API and re-install ?

https://cwiki.apache.org/confluence/display/AMBARI/Using+APIs+to+delete+a+service+or+all+host+compon...

[#4]

Hi kuldeep,

Could you provide more information on how to remove and re-install. When you mention re-install, do I have to disable kerberos from the ambari gui and enable? I am newbie to bigdata.

[#5]

@Kuldeep Kulkarni

[#6]

@sunil kanthety - Can you please try below command to delete KERBEROS_CLIENT

curl -H "X-Requested-By:ambari" -u admin:admin -X DELETE "http://<AMBARI-SERVER>:8080/api/v1/clusters/services/KERBEROS/components/KERBEROS_CLIENT

To re-install, again you can use Ambari API however easiest method is try to disable and re-enable Kerberos from Ambari UI.

Please do let me know how it goes.

[#7]

Kuldeep,

Should I run the CURL command on the host running the ambari-server or run it on all the other nodes individually. Could you please let me know.

[#8]

Kuldeep,

I ran the above CURL command on the host running the ambari-server and then tried to disable kerberos from ambari UI. The stop services step failed on all hosts. The stderr file has the following error:

Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 77, in <module> KerberosClient().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 31, in install self.configure(env) File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 40, in configure self.setup_jce() File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_common.py", line 448, in setup_jce raise Fail("Security is enabled, but JCE policy zip is not specified.") resource_management.core.exceptions.Fail: Security is enabled, but JCE policy zip is not specified.

[#9 · Accepted solution]

@sunil kanthety

You seem to be in an interesting state. What version of Ambari are you running?

For the issue you posted where the error is "Security is enabled, but JCE policy zip is not specified". Check your ambari.properties file for a property named "jce.name". The file listed there should exist in /var/lib/ambari-server/resources. If the property or the file does not exist we need to investigate why. For starters, what version of Java are you using and did you have ambari-server setup install it or did you install Java manually?

When enabling Kerberos via the UI. Does a button allowing you to skip the failed stage appear? If so, you should be able to click that and continue with the disable process. Once you complete disabling Kerberos, you should make sure that the Kerberos service is removed by issuing the following REST API call:

DELETE /api/v1/clusters/CLUSTERNAME/services/KERBEROS 

For example, using curl on a cluster named C1 where the administrator username and password have not be changed from the default values:

curl -H "X-Requested-By:ambari" -u admin:admin -X DELETE "http://<AMBARI-SERVER>:8080/api/v1/clusters/C1/services/KERBEROS

After this, you should be able to retry enabling Kerberos again.

[#10]

Thanks Robert for your inputs. I added the property jce.name in the ambari.properties file and restarted the ambari-server. After that the dekerberize options went through till the point that now it fails at the step "start services". From the error logs it indicates the following:

File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call raise Fail(err_msg) resource_management.core.exceptions.Fail: Execution of 'cat /var/lib/ambari-agent/tmp/pass | ACCUMULO_CONF_DIR=/usr/hdp/current/accumulo-tracer/conf/server /usr/hdp/current/accumulo-client/bin/accumulo shell -u root -f /var/lib/ambari-agent/tmp/cmds' returned 1. Password: ******** 2016-04-18 10:22:20,606 [trace.DistributedTrace] INFO : SpanReceiver org.apache.accumulo.tracer.ZooTraceClient was loaded successfully. 2016-04-18 10:22:20,677 [shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_CREDENTIALS for user root - Username or Password is Invalid

On doing some research found the following at the below link:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_releasenotes_ambari_2.1.0.0/content/amba...

followed the steps, but it indicated that "Accumulo is not properly configured."

I ran the bootstrap_config.sh file and chose the following options but it now fails indicating that JAVA_HOME is not set. I set the JAVA_HOME globally and in the accumulo-env.sh file. Still the error prevails.

Can you please let me know if i am doing it right.

Thanks