Running shell scripts in oozie using hue

[#1]

Running the below script:

#!/bin/bash
ssh user@host.abcd.com "if [ -f /data/home/user/datafiles/Test.csv ] ; then echo 1 ; else echo 0 ; fi;"

Set up passwordless ssh for 'user' too. 

Getting the below error:

Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]

Please Assist!

[#2 · Accepted solution]

This is probably related to the fact that the shell action, when run from Oozie, runs as user "yarn" and not as the desired user you're specifying in the ssh command

You can refer to this thread for more information about the issue:

https://community.cloudera.com/t5/Batch-Processing-and-Workflow/How-to-run-Oozie-workfllow-or-action...

It should all boil down (in case your cluster is not secured with kerberos) to try and set up your environment, specifically the "linux-container-executor" configuration parameter (you go in Cloudera Admin UI --> Yarn --> Configuration).

It's all explained in the linked document. Another alternative could be to grant OS user "yarn" with permissions to execute "ssh" and/or "su" so you can eventually switch user in your script before executing the remote ssh command

HTH

[#3]

So,

1.  I should uncheck thie below?

Limit Nonsecure Container Executor Users --> YARN (MR2 Included) (Service-Wide) { currently the check box is selected}

2. what should I put instead of nobody for the below?

UNIX User for Nonsecure Mode with Linux Container Executor

yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user

YARN (MR2 Included) (Service-Wide)  ---> Nobody 

3. Is there something more I should change?

4. Kerberos is not enabled in the system.

[#4]

1) Apparently, yes

2) The name of the user you're trying to use to log in to the remote system, I suppose. Pls note that the user you specify here would be the user "oozie" will run as, so you'd eventually get other problems of unpredictable nature when using Oozie

3) I don't really know, sorry about that... The fact is that even if I'm pretty sure to have understood the cause of your issue, I never had to deal with it directly myself. Maybe the easier way could be to follow the additional suggestions I wrote in my previous answer (give permissions to OS User "yarn" to "ssh" and/or "su"). Or, maybe, another possibility would be for you to create a "yarn" user on the remote system and grant this user with the correct permissions to get to the final working directory

I hope you'll manage to get through the problems and make it 🙂