execute shell commands in a set of nodes or all nodes

[#1]

I installed Cloudera Manager Express 15.5, and everything is working pretty fine.

However, I would like to be able to execute shell commands in the whole nodes (or a subset of nodes), without having to login to each individually and execute the command.

Is there a solution with Cloudera to do this?

Is there a way, using Manager, to enter a shell command that will get executed in the nodes we choose?

If not, what could be a solution?

Thanks!

[#2]

Hi @mdjedaini

I'm not aware of of such functionlity in cloudera manager, but you can do it using shell using something like this.

If you have a node that can ssh to all these nodes then you can issue a loop shell command:

for host in `cat /tmp/file_name`;do ssh $host bash -c 'hostname ; the shell_command you want to run ; done

If you don't have a node then you need to key keys to be able to run the mentioned command, you can do this using:

choose one of the nodes and generate public/privateusing by running the following command:

ssh-keygen
you will be asked for enter file in which to save the key and enter passphrase and enter passphrase again, in all these steps just press enter without taking any action.

then run from this node ssh-copy-id node2 and it will ask you for your password, once you enter it then you can directly ssh node2 from the node you chose.

You need to do the same for all the nodes in you cluser ( ssh-copy-id ).

in the node you choose add file using vim file_name and add the nodes you want to run the shell from.

then the run the command you want using:

for host in `cat /tmp/file_name`;do ssh $host bash -c 'hostname ; the shell_command you want to run ; done

[#3]

You can probably use ansible - which is very simple and straight forward . Let me know if you need more info on em 

[#4]

I actually never used it and yes if you could give some directions it would be great!

[#5]

@mdjedaini

There is nothing to do with cloudera on this request as there are so many other tools are available in the market. I am not sure how big your environment. In general, those who are using big environments with huge nodes will use some tools like Chef, Puppet, Terraform, Ansible, etc to achieve your requirement (for cloud there are another different set of tools like Cloudformation, etc)

In high level, you can divide them into two category: Push and Pull based

a. Tools like Puppet and Chef are pull based. Agent/Client on the server periodically checks for the configuration information from central server(master)

b. Ansible is Push based. Central server pushes the configuration information on target servers. You control when the changes are made on the servers