Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Oozie marks SSH Action Completed, but the command in SSH action is still in progress

avatar
New Contributor

For this action:

<action name="ssh-sas">
<ssh xmlns="uri:oozie:ssh-action:0.2">
<host>${user}@${host}</host>
<command>command for executing a SAS file</command>
<capture-output />
</ssh>
<ok to="success" />
<error to="failure" />
</action>

 

 

Oozie error encountered:

org.apache.oozie.action.ActionExecutorException: COULD_NOT_CONNECT: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 user@host mkdir -p oozie-oozi/0079310-190719160346834-oozie-oozi-W/ssh-sas--ssh/ ] | ErrorStream: Connection timed out during banner exchange

at org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:567)
at org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:206)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:232)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:459)
at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:82)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.command.XCommand.call(XCommand.java:356)
at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:280)
at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:61)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 user@host mkdir -p oozie-oozi/0079310-190719160346834-oozie-oozi-W/ssh-sas--ssh/ ] | ErrorStream: Connection timed out during banner exchange

at org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:342)
at org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:375)
at org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:208)
at org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
at org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:550)
... 17 more

 

what might be the issue here?

2 REPLIES 2

avatar
Master Collaborator

Per https://serverfault.com/questions/647569/reverse-ssh-connection-time-out-during-banner-exchange this error may indicate network issues. If you see a socket is established using netstat on both the server and the client, there may be a firewall or packet inspection device that is preventing the SSH connection from being established

 

Are you able to manually ssh from oozie server machine to the remote <host> as <user> 

avatar
New Contributor

The SSH connection is established initially at the beginning of the action and the <command> tag executes the command passed within the tags, but it takes a while(~ around 3 hours) for the command to execute, and in the meantime this action gets updated to COMPLETED and proceeds on to the next subsequent action, whereas former command is still in execution phase.

 

This exception arises when Oozie starts on the latter SSH action while the prior SSH action is still in execution.

 

So why does Oozie presumes the prior SSH action as completed yet the command is still in process?