Member since
11-29-2016
17
Posts
2
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1092 | 12-13-2016 09:36 PM | |
617 | 11-30-2016 03:37 PM |
07-27-2017
01:02 AM
1 Kudo
Typically you either need to scale out due to HDFS disk usage, or you need to scale out for computational reasons. If I have 10 or so datanodes and they are each allocated 80% of the system memory for YARN, would them all running 100% of their YARN allocation for a majority of the day indicate that I need to scale out for computational reasons? Currently only my HDFS is at 60% utilization. I am primarily running Tez jobs, CPU doesn't seem to be hit as much, but my YARN memory allocation is constantly 100% and I have users complaining about slow running jobs. I assume this is because they have to wait for other jobs to free up resources for them to get their job to run. Are there any things I could look for in this situation? Running Ambari 2.5.1 and HDP 2.6.1.
... View more
Labels:
- Labels:
-
Apache YARN
01-25-2017
06:40 PM
@apappu Sorry, I was just able to get to the office to try this out. This was the issue, thanks for your help!
Just out of curiosity, why can't a non-root user use port 443?
... View more
01-24-2017
10:13 PM
Hello, I enabled HTTPS for my Ambari Server before I changed it to run as a non-root daemon user. After I enabled non-root daemon, I'm getting the following error: 24 Jan 2017 17:06:48,001 WARN [main] AbstractLifeCycle:204 - FAILED SslSelectChannelConnector@0.0.0.0:443: java.net.SocketException: Permission denied
java.net.SocketException: Permission denied
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316)
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265)
at org.eclipse.jetty.server.ssl.SslSelectChannelConnector.doStart(SslSelectChannelConnector.java:631)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.Server.doStart(Server.java:293)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.apache.ambari.server.controller.AmbariServer.run(AmbariServer.java:617)
at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:927)
It seems as though even though I've put in all the sudo settings (starting here: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-security/content/commands_server.html ) the non-root user still doesn't have enough permissions to read the certificate to use for SSL binding. Does anyone know what is needed for this permission issue to be resolved? The SSL certificate and key are installed in /etc/ssl/certs/ I've been searching and I can't seem to find an answer to this. Thanks
... View more
Labels:
- Labels:
-
Apache Ambari
01-19-2017
03:33 PM
@Oliver Fletcher Yup, this was the issue. I enabled LDAPS on our domain and it works now.
... View more
01-11-2017
07:38 PM
@lraheja
Sure, it's no longer timing out, it's just back to what it was doing before.
kerberos-stack-2.txt
... View more
01-11-2017
06:26 PM
@lraheja I did not go through the ambari-server setup-ldap steps, I must've gone past this some how. After configuring this and restarting Ambari the LDAP tests seem to be getting further but are now just timing out.
My krb5.conf is not configured at all, it's the default conf file. I assumed Ambari was going to configure this through the wizard, is that not the case?
... View more
01-11-2017
06:03 PM
Hi @rguruvannagari thanks for the reply.
I just confirmed with my AD guy that our AD is not set up for SSL at all. I was able to parse AD using the ldapsearch tool using the same DN and ldap url I'm specifying. I'll keep trying different DN's
... View more
01-11-2017
05:39 PM
Hello, I'm receiving this error: Failed to connect to KDC - Failed to communicate with the Active Directory at LDAP://hq.domain.com/OU=Production,OU=domain,DC=hq,DC=domain,DC=com: simple bind failed: hq.domain.com:389
Update the KDC settings in krb5-conf and kerberos-env configurations to correct this issue.
I've been following this guide: https://www.ibm.com/support/knowledgecenter/SSPT3X_4.2.0/com.ibm.swg.im.infosphere.biginsights.admin.doc/doc/admin_kerb_activedir.html as well as the HDP documentation on this. I'm doing the automated kerberos wizard. JCE has been distributed to all of the nodes, I'm using Oracle JDK 1.8. Attached is the full stack trace: kerberos-stack.txt The KDC Test Connection passes just fine, I can see expected network traffic between my domain controller and the Ambari server. The only main difference is that I'm not using SSL on AD. I figure this should be fine and Ambari can just use the plaintext 389 port. I realize this is a security concern but I have no way around this right now. I don't have control over this area of our domain setup. Could this be my issue? Any help appreciated. Thanks.
EDIT: I was able to successfully parse AD using the ldapsearch tool using the same DN and LDAP url that I'm specifying. Also with the same admin user.
... View more
Labels:
12-13-2016
09:36 PM
1 Kudo
This is resolved. There were a couple directories leftover in the /usr/hdp location using the old version, it seems Ambari will use this file path to determine the version needed, not sure how to articulate this further but yeah, something metadata wise was changed based on these files existing still. Also doing "python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent" on every machine helped clean up things that were missed. I forgot to run this step.
... View more
12-13-2016
04:51 AM
Hi @rgangappa Thanks for the reply. Unfortunately I've tried this already and it didn't fix the issue. I don't believe the HDP.repo file is the problem, the correct repo is being used, it's just Ambari is instructing yum to install the wrong package - - a package in which does not exist in the correct repo that is loaded into yum.
... View more