question Re: Error During adding a new new node into cluster in Archives of Support Questions (Read Only)

Error During adding a new new node into cluster

Learner_1122 — Sun, 18 Aug 2019 00:48:12 GMT

Hi Guys,
I have been facing an issue.
I have been using AWS Hortonworks machine.Firstly i am successfully login into ambari and it shows all services run.BUT when i try to add another node into cluster. it give me an error :

then i close the browser and try to re-connect the error. but now it give me an error :
"Unable to Connect Ambari "
whe i try to write command :
" ambari-server start"
the it give me message "Command not found"
Please help
Thanks

Re: Error During adding a new new node into cluster

nshetty — Tue, 17 Oct 2017 21:44:33 GMT

@Mudassar Hussain Are you trying "ambari-server start" in the correct host where ambari server is installed? As you are saying you were able to login to ambari UI earlier.

Re: Error During adding a new new node into cluster

Learner_1122 — Thu, 19 Oct 2017 13:01:39 GMT

@nshetty

thanks for you reply.I have solve this issue by writing the command in correct host @namenode and it show Ambari Server Start. when i try to login to Ambari it give me same error.
Most of people saying you need to create a new instance. but after 15 minutes, ambari again stop. I am stuck at this point.

Re: Error During adding a new new node into cluster

Learner_1122 — Thu, 19 Oct 2017 13:11:55 GMT

thanks for your prompt reply @Aditya Sirna
When i try to write this command it does not show me the log. a message appeared "unknow command".
I think i am not writing the command on right host as @nshetty said.
I am new to this.
How i confirm that on which node ambari server install ? i tried these command on @namenode

Re: Error During adding a new new node into cluster

jsensharma — Thu, 19 Oct 2017 14:46:59 GMT

@Mudassar Hussain

Following steps you can try to findout what might be wrong:

On the Host where "ambari-server" is installed try the following:

SSH to ambari server host (using IP address or using Hostname)

1). Check if the ambari-server is running and listening to port 8080 (default port)

# netstat -tnlpa | grep 8080

2). If the Port is not listening then please check the following log to see if there are any error logged. If any errors are logged then please share the logs.

# less /var/log/ambari-server/ambari-server.log
# less /var/log/ambari-server/ambari-server.out

3). If there are no errors logged then try restarting the ambari server and also varify if the Hostname for the ambari server is setup correctly and the Iptables (Firewall) is disabled.

# hostname -f
# cat /etc/hosts
# service iptables stop  (OR  for CentOS7 the command will be  "systemctl disable firewalld")
# ambari-server restart

If you still do not find the "ambari-server" command then please verify if that host has the ambari packages installed? (just to be double sure that you logged in to correct ambari-server host)

# ls -l /usr/sbin/ambari-server
# rpm -qa | grep ambari
(OR)
# rpm -qa | grep ambari-server

4). After restarting the abari-server as following please check if the port 8080 is opened or not? (verify the ambari-server.log as well, to see no errors logged)

# ambari-server start
# less /var/log/ambari-server/ambari-server.log
# netstat -tnlpa | grep 8080

From the machine where you are running the browser

5). If ambari-server port (8080) is opened and listening then from the machine where you are running the browser please check if the port is accessible or not?

# nc  -v  $AMBARI_SERVER_IP  8080
(OR)
# telnet  $AMBARI_SERVER_IP  8080

6). If the port is accessible then try to access ambari UI from the browser (if needed then use Incognito Mode on your browser to isolate the browser caching issue)

On All Agent Hosts

On All the ambari agent machines please make sure tthe following things are OK:

1. The iptables is stopped and the Hostname (FQDN) is properly setup,

2. Password less SSH is setup (if you are planning to register the new agents from the Ambari UI). Means from ambari-server host you can do passwordless ssh to the agent machines.

3. The Agent machines has proper internet connectivity, to download packages from HWX repo When instructed by ambari server.

4. The /etc/hosts file has proper entry on all agent machines to resolve the ambari-server hostname properly. And from agent machines there is no issues in connecting to the following port of ambari server:

# nc  -v  $AMBARI_SERVER_HOSTNAME  8080
# nc  -v  $AMBARI_SERVER_HOSTNAME  8440
# nc  -v  $AMBARI_SERVER_HOSTNAME  8441

Re: Error During adding a new new node into cluster

Learner_1122 — Sun, 18 Aug 2019 00:48:05 GMT

Got your Point regarding Ambari host.
my ambari URL is : namenode:8080 so i log into @namenode and wrie the command " ambari-server restart"
this command successfully run and it shows "Ambari Server Run Successfully".
when i try to get the log by writing below command :
/var/log/ambari-server/ambari-server.log
it gives me message : "Permission Denied"
see the attached image

Re: Error During adding a new new node into cluster

jsensharma — Thu, 19 Oct 2017 14:51:06 GMT

@Mudassar Hussain

Looks like ambari-server is started fine now. Just to be sure please check the logs of ambari as following and verify the port is accessible:

# less /var/log/ambari-server/ambari-server.log
# netstat -tnlpa | grep 8080
# service iptables stop

Then try accessing the ambari using the mentioned IP Address:

http://$AMBARI_SERVER_IP:8080

Re: Error During adding a new new node into cluster

jsensharma — Thu, 19 Oct 2017 14:54:08 GMT

@Mudassar Hussain

Regarding the

/var/log/ambari-server/ambari-server.log 
it gives me message : "Permission Denied"

This is because you need to specify the commands like "vi" , "cat" or "less" before the filename as following:

# less /var/log/ambari-server/ambari-server.log 
# vi /var/log/ambari-server/ambari-server.log 
# cat /var/log/ambari-server/ambari-server.log

Re: Error During adding a new new node into cluster

Learner_1122 — Sun, 18 Aug 2019 00:47:57 GMT

Thanks a lot @Jay SenSharma
Two Hours Before i wrote above 3 command. Ambari server Start and I was so happy that time 🙂
but Machine was not working properly so i create the new AWS instance.
Now i have facing same issue : Ambari Server not login 😞
although i wrote these 3 lines.
One Question How i export the log which i see after this command :
# less /var/log/ambari-server/ambari-server.log
see the attached screen

Re: Error During adding a new new node into cluster

jsensharma — Thu, 19 Oct 2017 18:47:38 GMT

@Mudassar Hussain

Can you please attach the "ambari-server.log" to this HCC thread.

You can download/export the "/var/log/ambari-server/ambari-server.log" file from your AWS instance to your local machine using SCP commands like:

# scp  -i ~/.ssh/YOUR_AWS_KEY  root@namenode:/var/log/ambari-server/ambari-server.log   /tmp

Re: Error During adding a new new node into cluster

Learner_1122 — Sun, 18 Aug 2019 00:47:49 GMT

My AWS key is : AWS_HDPCD
When i write the above command it gives me below message :

Re: Error During adding a new new node into cluster

Learner_1122 — Thu, 19 Oct 2017 19:54:54 GMT

I got the log file. Machine is too much slow. i try to email myself there.

Re: Error During adding a new new node into cluster

jsensharma — Thu, 19 Oct 2017 20:09:14 GMT

@Mudassar Hussain

I see that you are getting "No Such file or directory" error. So first you will need to find the AWS ssh key on your local machine to do a successful SCP.

Or if the SSH key is already setup then you can simply run the command without -i option as following to see if it works:

# scp root@namenode:/var/log/ambari-server/ambari-server.log   /tmp

Re: Error During adding a new new node into cluster

Learner_1122 — Thu, 19 Oct 2017 20:16:16 GMT

I got the log file. Machine is too much slow. i try to email myself there.

Re: Error During adding a new new node into cluster

Learner_1122 — Wed, 25 Oct 2017 14:58:21 GMT

Thanks a lot @Jay SenSharma for your help.
Today when i was login to my AWS machine. it does not start all service and give me an error 8080:Port and Ambari Server issue.
I just Go inot @nameNode
and write command : Service Iptables Stop
and both issue solves 🙂 🙂
Hope it will work smoothly.
Thanks again