https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/webapp/src/main/webapp/oozie-console.js#L384  ... View more

@A C Just to understand, did you run the spark submit using yarn cluster as master/deploy mode? If so, let's try to check the job properties for the following parameter: ${resourceManager} Also, here it is another example regarding pyspark + oozie (using shell to submit spark). https://github.com/hgrif/oozie-pyspark-workflow Hope this helps ... View more

@Felix Albani i am facing the issues while i am running the job in oozie . Below is the error 2018-08-21 14:27:16,632 ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.IllegalArgumentException: Invalid ContainerId: container_e63_1532612094367_17160_02_000001 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1343) ... View more

Yaeh,I don't konw why,I change another cluster,every thing is fine.so thank you very much! ... View more

Ok,thank you very much.I thought that in the fact there is a way to remove it,just I don't know。 ... View more

You're very welcome! Yes, for this the setrep command can change the replication factor for existing files and there is no need to change the value globally. For others' reference, the command is: hdfs dfs -setrep [-R] [-w] <numReplicas> <path> ... View more

thank you !I will try this. ... View more

Thank you ... View more