Support Questions

Find answers, ask questions, and share your expertise

Best Pratices to develop with HUE workflow and pyspark in HDP 2.4

I'm using HDP 2.4 and HUE.

I want to develop use cases using pyspark and hue workflow to automate/schedule the pyspark scripts, what are the best practices?

I tried to launch a spark-submit command trow hue workflow(oozie action) shell module because the spark module seams to not work on my hdp version.

But I can't run it because seams that the shell module don't find my pyspark file, but i have confirmed that the file is in the container.

Can someone help me, thanks