Member since
03-22-2018
7
Posts
1
Kudos Received
0
Solutions
01-04-2019
08:50 PM
Hello @Akhil S Naik thanks for your reply. I did query host_role_commands and unfortunately, i only got "ABORTED" ,"COMPLETED" and "FAILED" states. What's more interesting is that the biggest task_id on that table is "977". Idon't really know where that 1546517218 on the logs is coming from. Kindly let me know if you have additional thoughts, please.
... View more
01-03-2019
09:06 PM
I have an HDP-3.1.0.0, Ambari-managed cluster on AWS. I just successfully ran the Kerberos wizard, as well as the sync-ldap command from the CLI with an existing Active Directory. After completing the UI Kerberos Wizard, The zookeeper service on the master node failed to start. After that, i can manually start the services on some of the nodes, but not others. Once i issue the start/restart command from the UI, the operation will get stuck at 9% and it will not show any output.
After googling a bit, i found this question, and tried all suggested fixes (memory is more than enough, and there are no STOPPING hostcomponentstate rows). I restarted the agent and server processes on all nodes. Even issues a reboot on the whole cluster (Services are set to auto-start). When back on the UI, some services will be up but others will not. Issuing start commands work for a while. After one fails, it will keep failing and stop showing progress/output.
Looking at ambari-server logs, i get these suspicious warnings:
2019-01-03 19:08:09,423 WARN [agent-report-processor-0] ActionManager:155 - The task 1546517134 is invalid
2019-01-03 19:08:09,680 WARN [agent-report-processor-0] HeartbeatProcessor:358 - Can't fetch HostRoleCommand with taskId = 1546517135
2019-01-03 19:08:09,680 WARN [agent-report-processor-0] ActionManager:155 - The task 1546517135 is invalid
2019-01-03 19:08:15,989 WARN [agent-report-processor-2] HeartbeatProcessor:358 - Can't fetch HostRoleCommand with taskId = 1546517218
2019-01-03 19:08:15,990 INFO [agent-report-processor-2] ServiceComponentHostImpl:1054 - Host role transitioned to a new state, serviceComponentName=RANGER_TAGSYNC, hostName=worker3.devbigdata.spendhq.net, oldState=INSTALLED, currentState=STARTED
2019-01-03 19:08:15,991 WARN [agent-report-processor-2] ActionManager:155 - The task 1546517218 is invalid
2019-01-03 19:08:16,207 WARN [agent-report-processor-2] HeartbeatProcessor:358 - Can't fetch HostRoleCommand with taskId = 1546517219
2019-01-03 19:08:16,207 WARN [agent-report-processor-2] ActionManager:155 - The task 1546517219 is invalid
And ambari agents on all hosts show lines like these: INFO 2019-01-03 19:25:37,747 __init__.py:82 - Event from server at /user/ (correlation_id=4683): {u'status': u'OK', u'id': 2510}
INFO 2019-01-03 19:25:38,521 ComponentStatusExecutor.py:183 - Status for DATANODE has changed to INSTALLED
INFO 2019-01-03 19:25:38,521 RecoveryManager.py:174 - current status is set to INSTALLED for DATANODE
INFO 2019-01-03 19:25:38,728 security.py:135 - Event to server at /reports/component_status (correlation_id=4684): {'clusters': defaultdict(<function <lambda> at 0x7f9a3c569410>, {u'2': [{'status': 'INSTALLED', 'componentName': u'ZOOKEEPER_SERVER', 'serviceName': u'ZOOKEEPER', 'clusterId': u'2', 'command': u'STATUS'}, {'status': 'INSTALLED', 'componentName': u'SECONDARY_NAMENODE', 'serviceName': u'HDFS', 'clusterId': u'2', 'command': u'STATUS'}, {'status': 'INSTALLED', 'componentName': u'DATANODE', 'serviceName': u'HDFS', 'clusterId': u'2', 'command': u'STATUS'}]})}
INFO 2019-01-03 19:25:38,730 __init__.py:82 - Event from server at /user/ (correlation_id=4684): {u'status': u'OK'} Any help is greatly appreciated!
... View more
Labels:
- Labels:
-
Apache Ambari
12-18-2018
01:53 AM
I have a HDP 3.0, Ambari 7 cluster. When running the kerberos Wizard, I am running into an issue where the test kerberos ticket created by the wizard haves an old, incorrect domain i inputted by mistake. After some research, it seems to be an issue with the UI storing the old Domain Value. I tried the suggested steps on this thread without luck: https://community.hortonworks.com/questions/6437/when-kerberizing-via-ambari-against-an-ad-the-serv.html I also found the value was being stored on Chrome's LocalStorage, so i cleared it. After a second try, it haves the correct value, but the keytab file keeps ending up wrong. Is there a way to make sure that value gets reset properly? I'm thinking on destroying the cluster and starting anew at this point. Thanks so much!
... View more
Labels:
- Labels:
-
Apache Ambari
12-13-2018
03:20 PM
Thanks for your quick reply @Gulshad Ansari With your suggestion, i was able to understand a little better my scenario. I need to use my AD domain for the admin principal, which i was not. Anyways, when doing the change, i ran into an issue, where the test kerberos ticket created by the wizard still haves the old, incorrect domain. I did some quick research, and it seems to be an issue with the UI storing the old Domain Value. I tried the suggested steps on this thread without luck: https://community.hortonworks.com/questions/6437/when-kerberizing-via-ambari-against-an-ad-the-serv.html I also found the value was being stored on Chrome's LocalStorage, so i cleared it. After a second try, it haves the correct value, but the keytab file keeps ending up wrong. Is there a way to make sure that value gets reset properly? I'm thinking on . destroying the cluster and starting anew at this point. Thanks so much!
... View more
12-07-2018
09:15 PM
I got an HDP 3.0 Ambari 2.7 Cluster on AWS EC2, which i'm trying to integrate with a Company LDAP. Been having issues when trying to continue on the last step, when the wizard Tests Kerberos Connection, it asks me for the Admin principal and password. When doing that, i get an error on the test request (after the Credentials PUT request). When looking at the logs, i see: javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C09042F, comment:
AcceptSecurityContext error, data 52e, v2580]
I also tried to test connection to the LDAP using ambari-server sync-ldap. After following the wizard, i was able to connect and get all users on the container. I'm assuming this utility does not share some settings with the UI. If that is the case, what can i do to fix it? My setup is attached captura-de-pantalla-2018-12-07-a-las-40029-p-m.png captura-de-pantalla-2018-12-07-a-las-40334-p-m.pngcaptura-de-pantalla-2018-12-07-a-las-40452-p-m.png
... View more
Labels:
- Labels:
-
Apache Ambari
10-29-2018
03:17 PM
1 Kudo
Incredible. Ambari 2.7 is useless as it is, then. Just ODBC...
... View more