Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

execute shell commands in a set of nodes or all nodes

avatar
New Contributor

I installed Cloudera Manager Express 15.5, and everything is working pretty fine.

 

However, I would like to be able to execute shell commands in the whole nodes (or a subset of nodes), without having to login to each individually and execute the command.

 

Is there a solution with Cloudera to do this?

 

Is there a way, using Manager, to enter a shell command that will get executed in the nodes we choose?

 

If not, what could be a solution?

 

Thanks!

4 REPLIES 4

avatar
Master Collaborator

Hi @mdjedaini

 

I'm not aware of of such functionlity in cloudera manager, but you can do it using shell using something like this.

 

If you have a node that can ssh to all these nodes then you can issue a loop shell command:

 

for host in `cat /tmp/file_name`;do ssh $host bash -c 'hostname ; the shell_command you want to run ; done

 

 

If you don't have a node then you need to key keys to be able to run the mentioned command, you can do this using:

 

choose one of the nodes and generate public/privateusing by running the following command:

ssh-keygen
you will be asked for enter file in which to save the key and enter passphrase and enter passphrase again, in all these steps just press enter without taking any action.

 

then run from this node ssh-copy-id node2 and it will ask you for your password, once you enter it then you can directly ssh node2 from the node you chose.

 

You need to do the same for all the nodes in you cluser ( ssh-copy-id ).

in the node you choose add file using vim file_name and add the nodes you want to run the shell from.

 

then the run the command you want using:

 

for host in `cat /tmp/file_name`;do ssh $host bash -c 'hostname ; the shell_command you want to run ; done

avatar
Champion

You can probably use ansible - which is very simple and straight forward . Let me know if you need more info on em 

avatar
New Contributor
I actually never used it and yes if you could give some directions it would be great!

avatar
Champion

@mdjedaini

 

There is nothing to do with cloudera on this request as there are so many other tools are available in the market. I am not sure how big your environment. In general, those who are using big environments with huge nodes will use some tools like Chef, Puppet, Terraform, Ansible, etc to achieve your requirement (for cloud there are another different set of tools like Cloudformation, etc)

 

In high level, you can divide them into two category: Push and Pull based

a. Tools like Puppet and Chef are pull based. Agent/Client on the server periodically checks for the configuration information from central server(master)

b. Ansible is Push based. Central server pushes the configuration information on target servers. You control when the changes are made on the servers