Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oozie marks SSH Action Completed, but the command in SSH action is still in progress

Oozie marks SSH Action Completed, but the command in SSH action is still in progress

New Contributor

For this action:

<action name="ssh-sas">
<ssh xmlns="uri:oozie:ssh-action:0.2">
<host>${user}@${host}</host>
<command>command for executing a SAS file</command>
<capture-output />
</ssh>
<ok to="success" />
<error to="failure" />
</action>

 

 

Oozie error encountered:

org.apache.oozie.action.ActionExecutorException: COULD_NOT_CONNECT: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 user@host mkdir -p oozie-oozi/0079310-190719160346834-oozie-oozi-W/ssh-sas--ssh/ ] | ErrorStream: Connection timed out during banner exchange

at org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:567)
at org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:206)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:232)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:459)
at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:82)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:280)
at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:61)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 user@host mkdir -p oozie-oozi/0079310-190719160346834-oozie-oozi-W/ssh-sas--ssh/ ] | ErrorStream: Connection timed out during banner exchange

at org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:342)
at org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:375)
at org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:208)
at org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
at org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:550)
... 17 more

 

what might be the issue here?

2 REPLIES 2
Highlighted

Re: Oozie marks SSH Action Completed, but the command in SSH action is still in progress

Expert Contributor

Per https://serverfault.com/questions/647569/reverse-ssh-connection-time-out-during-banner-exchange this error may indicate network issues. If you see a socket is established using netstat on both the server and the client, there may be a firewall or packet inspection device that is preventing the SSH connection from being established

 

Are you able to manually ssh from oozie server machine to the remote <host> as <user> 

Highlighted

Re: Oozie marks SSH Action Completed, but the command in SSH action is still in progress

New Contributor

The SSH connection is established initially at the beginning of the action and the <command> tag executes the command passed within the tags, but it takes a while(~ around 3 hours) for the command to execute, and in the meantime this action gets updated to COMPLETED and proceeds on to the next subsequent action, whereas former command is still in execution phase.

 

This exception arises when Oozie starts on the latter SSH action while the prior SSH action is still in execution.

 

So why does Oozie presumes the prior SSH action as completed yet the command is still in process?

Don't have an account?
Coming from Hortonworks? Activate your account here