Created 05-22-2017 05:57 PM
Dear all,
I am working on a cluster with several VM and I need to run some pyspark code through Oozie periodically on a specific cluster machine, however I am not able to find a configuration that allows me to do that.
My workarround so far is to run a ssh client session with oozie that will spark-submit the script. Is this the only way?
Thanks in advance
Created 06-01-2017 05:59 PM
Actually you are not able to force the oozie action running in a certain node unless you are using ssh action. Once the Oozie submit the action to Yarn, Yarn will running the job in one of the NodeManger host.
Created 06-01-2017 05:59 PM
Actually you are not able to force the oozie action running in a certain node unless you are using ssh action. Once the Oozie submit the action to Yarn, Yarn will running the job in one of the NodeManger host.
Created 06-03-2017 04:40 PM
I will then keep it working with the ssh session.
Thank you very much for your feedback.