Member since
09-01-2018
17
Posts
0
Kudos Received
0
Solutions
10-05-2021
01:27 AM
I use OOZIE to run a workflow. But a simple official example shell-wf (echo hello oozie) stuck in RUNNING state and never end. The workflow can be submitted but stuck at RUNNING state. There is not any error in job log in OOZIE UI. When submitting a shell with spark-submit inside, the job will be never submitted and can not be seen in Spark UI. I suspect the shell didn't run at all. customer survey
... View more
10-17-2019
08:17 AM
Run Hive Jobs with Oozie Specify the hive-site.xml in the job-xml parameter. Specify the name of the script (for example, script.q ) that contains the hive query in the script parameter. Optionally, add properties used by the Oozie launcher job. Add the prefix oozie.launcher to the property names. see here Dgcustomefirst
... View more
09-27-2018
01:17 PM
1 Kudo
@A C To change the format of the input time we need to use unix_timestamp along with from_unixtime funcitons. Try with below syntax hive> select from_unixtime(unix_timestamp("Tue Sep 26 22:02:11 CDT 2018",'EEE MMM dd HH:mm:ss z yyyy'),'yyyy-MM-dd HH:mm:ss');
+----------------------+--+
| _c0 |
+----------------------+--+
| 2018-09-26 22:02:11 |
+----------------------+--+ - If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.
... View more
09-17-2018
05:46 AM
@A C Just to understand, did you run the spark submit using yarn cluster as master/deploy mode? If so, let's try to check the job properties for the following parameter: ${resourceManager} Also, here it is another example regarding pyspark + oozie (using shell to submit spark). https://github.com/hgrif/oozie-pyspark-workflow Hope this helps
... View more