Adding Clients to Sandbox via Ambari

[#1]

One of my engineers requested an HDP sandbox within vSphere be added and a secondary ‘client’ vm to connect with. Seemed like a simple enough request but I’m stumbling at the first step of adding the client via Ambari, which I’ve done hundreds if not thousands of times in the past.

Setup:

10.x.x.151 sandbox.hortonworks.com sandbox ambari.hortonworks.com

10.x.x.152 client.hortonworks.comclient

Passwordless ssh works between both systems:

[root@sandbox ~]# ssh client 'echo $HOSTNAME'

client.hortonworks.com

[root@sandbox ~]#

[root@client ~]# ssh sandbox 'echo $HOSTNAME'

sandbox.hortonworks.com

[root@client ~]#

JAVA was matched and home set, not that it matters because I’m unable to register

SELINUX / IPtables off

Are there restrictions within Sandbox that prevent you from adding additional hosts to the single cluster? I've manually built dozens of clusters but have yet to run into a problem like this in the beginning. Have not tried manually downloading packages and pulling in configs from the sandbox, but I would prefer to have everything managed by Ambari, if possible before going that route.

[#2]

@John Kitterman personally I would not use the sandbox to add multipule host. Sandbox is simply to test and learn functionality. Adding host would best be done on a non-sandbox environment.

[#3]

@Sunile Manjee

Yes, I completely agree with you and had that exact conversation with my engineer. In the end, he had his reasons and I ended up being tasked with the request.

Adding to this, I'm not planning to add anything like a datanode or hive database. He's only looking at 'clients' hooked into an existing infrastructure that include all of the bells and whistles to play with. Snap, break and restore.

[#4]

Here is the error, which leads me to believe passwordlesss ssh is not configured correctly, which is not the case

==========================
Creating target directory...
==========================

Command start time 2016-04-28 18:49:48

Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
SSH command execution finished
host=client.hortonworks.com, exitcode=255
Command end time 2016-04-28 18:49:48

ERROR: Bootstrap of host client.hortonworks.com fails because previous action finished with non-zero exit code (255)
ERROR MESSAGE: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

STDOUT:  

Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[#5]

Try using the manual steps that put in the answer. This is a workaround for your passwordless ssh issue. With passwordless ssh, check if your are putting the right key.

[#6 · Accepted solution]

Can you try manually installing ambari-agent on the client node using instructions here. Once you start ambari-agent, you can check ambari-server logs to see if client.hortonworks.com registered. As long as you got the network part figured out (like I see from your setup), there should be nothing else from sandbox preventing adding a new node.

[#7]

Sure, I'll try this when time permits.

[#8]

Added client via yum and updated configuration file with headend. Clearly saw 'client.hortonworks.com' in ambari-server.log. For giggles, I hit retry on registration and it worked on first time. What? All client services are installed. I'll consider this, well - a 'feature' of HDP2.4.0 and leave it at that.

Thanks for steering me towards this odd resolution.

[#9]

When you run into network issues with passwordless ssh, manual install of ambari-agent is your best bet. Once you manually register the node in ambari from ambari UI, then you can install components from UI as usual. Most likely you ran into some passwordless ssh quirks which can still be debugged looking at your ambari logs.