@Liana Napalkova The error is related to Port bind Failed to start namenode.java.net.BindException:Portinuse: xxxxxxmaster1.local.xxxxxx.org:50070 . So as mentioned earlier i will suggest you to first recheck the ports with root user to see if some other process i by any chance capturing those ports? If yes then you will need to kill those processes first. # netstat -tnlpa | grep 50070 # kill -9 $PID . If you see that there is no such process on your NameNode host which is using that port then please try thsi: 1. Reboot the Host (OS) just to isolate any Zombie process issue. 2. Now after the host reboot please try to start the NameNode and DataNode processes using command line to see if everything works fine. As mentioned in https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_reference/content/starting_hdp_services.html # su -l hdfs -c "/usr/hdp/current/hadoop-hdfs-namenode/../hadoop/sbin/hadoop-daemon.sh start namenode" . Mostly it can happen if the "hostname -f" command out put on your NameNode is same as the following properties in your "Advanced hdfs-site.xml" Ambari UI --> HDFS --> Configs --> Advanced --> "Advanced hdfs-site.xml" Properties to check: dfs.namenode.http-address AND dfs.namenode.rpc-address . Please make Sure that the FQDN of your NameNode is setup correctly and the entries in the "/etc/hosts" files are correctly having the hostname entry with the correct IPAddress mapping. ... View more

@Liana Napalkova Looks like it is failing because the HDFS service is not starting. Based on the Error it looks like the NameNode port 50070 is in conflict and hence NameNode is not starting because of the port conflict. 2018-04-09 12:37:44,761 ERROR namenode.NameNode (NameNode.java:main(1759)) - Failed to start namenode.java.net.BindException: Port in use: xxxxxxmaster1.local.xxxxxx.org:50070 at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:963) . . Caused by: java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.Net.bind(Net.java:425) . So please find the process that is listening on the port 50070 and then kill it and then try to start the HDFS service again. # netstat -tnlpa | grep 50070 Then kill the processID which is returned by the above command # kill -9 $PID . Also please check if you have defined the correct hostname (xxxxxxmaster1.local.xxxxxx.org) for your name node and if the port 8020 & 50070 are free on the host where you are trying to run the NameNode process. ... View more

@Liana Napalkova If this resolved your query then please mark this HCC thread as answered by clicking on "Accept" link on the correct answer, That way it will help other HCC users to quickly find the answers. ... View more

[SSL: CERTIFICATE_VERIFY_FAILED] the following doc provides more detailed information about the "certificate verify failed (_ssl.c" issue while using RHEL7: Controlling and troubleshooting certificate verification: https://access.redhat.com/articles/2039753#controlling-certificate-verification-7 ... View more

@Liana Napalkova When you setup Passwordless SSH then you will need to make sure that the passwordless ssh is set for the correct user. For example if you are running Ambari Server and Agents both as "root" user then you should setup the passwordless ssh for that same user. I will suggest you to please try this again: 1. Generate SSH keys on Ambari Server Host (Master). (While generating the keys please keep the "passphrase" empty) # ssh-keygen Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: 2. Now from ambari server host run the following command to setup the passwordless SSH from Master to all Slave hosts. # ssh-copy-id -i ~/.ssh/id_rsa root@slave1.example.com # ssh-copy-id -i ~/.ssh/id_rsa root@slave2.example.com 3. Now you should be able to test the passwordless SSH from ambari server to ambari agents. (not need to setup passwordless SSH from agent to host) 4. Now when in the ambari UI it asks for the Private key info then please use the content of file # cat /root/.ssh/id_rsa 5. The permission on the files should be something like this: (On Master Ambari Server) # ls -l /root/.ssh/id_rsa* -rw-------. 1 root root 1679 Mar 13 08:39 /root/.ssh/id_rsa -rw-r--r--. 1 root root 407 Mar 13 08:39 /root/.ssh/id_rsa.pub # ls -ld /root/.ssh drwx------. 2 root root 58 Mar 15 23:27 /root/.ssh 6. The permission on the files should be something like this: (On All the Slave Agent Hosts) # ls -l /root/.ssh/ -rw-------. 1 root root 819 Jun 9 2017 authorized_keys # ls -ld /root/.ssh drwx------. 2 root root 28 Jun 9 2017 /root/.ssh . The problem can be either due to the following facts: 1. If the "~/.ssh" and it's contents are not set correctly as mentioned above. 2. If the FQDN of every Host is not setup correctly. Please check the output of the following command on every host to see if the FQDN is setup correctly? https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-installation-ppc/content/edit_the_host_file.html # hostname -f 3. If the passwordless SSH is setup for the correct user. SELinux is disabled and Firewall is off on all nodes including ambari server host. . ... View more

@Siddharth Mishra Can you please try the following and share the logs: 1. Try starting the HDFS compoents manually to see if they are getting started or not? If not starting then please share the "/var/log/hadoop/hdfs/hadoop-hdfs-datanode-xxxx.log" and "/var/log/hadoop/hdfs/hadoop-hdfs-namenode-xxxx.log" logs. # su -l hdfs -c "/usr/hdp/current/hadoop-hdfs-namenode/../hadoop/sbin/hadoop-daemon.sh start namenode" # su -l hdfs -c "/usr/hdp/current/hadoop-hdfs-datanode/../hadoop/sbin/hadoop-daemon.sh start datanode" 2. Please share the output of the following directories to see if the permissions are setup as following on the directories? # ls -ld /hadoop/hdfs/data drwxr-x---. 3 hdfs hadoop 38 Apr 6 09:42 /hadoop/hdfs/data 3. Are you running your Ambari Agents/Server as Non Root User account? If yes then you should follow the doc to make sure that the permissions and sudoer configs are correctly set so that the ambari user can read the required directories. https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-security/content/configuring_ambari_for_non-root.html ... View more

@Shu Thank you for sharing so detailed and great answers as always. 😄 ... View more