Created on 03-31-2017 12:43 PM - edited 09-16-2022 04:23 AM
I set up my workflow put it into hdfs as well and I try to run the conf directory the properties file with this syntax.
I am really not sure why it is not working if I have a typo in my workflow.xml or job.properties or if I need to modify some config setting. Thanks
Error message
Here is the link to the error message, https://ibb.co/dkHnJv
Created 04-01-2017 09:44 AM
Pease show your job.properties file and your oozie submit command please.
A few base questions:
You might also try looking at the Oozie logs, which may give you more details on the error.
Created 04-02-2017 08:23 PM
Properties file # Environment settings queueName = default kerberos_realm = A jobTracker = B:8032 nameNode = hdfs://nameservice1 hive2_server = C hive2_port = 10000 impala_server = D:21000 edge_server = E jobTracker = yarnrm # Project specific paths projectPath = /user/${user.name}/oozie/mediaroom-logs keyTabLocation = /user/${user.name}/keytabs # job path oozie.wf.application.path = ${projectPath}/BXDB/wf # Project specific jars and other libraries oozie.libpath = ${projectPath}/lib,${projectPath}/util # Standard useful properties oozie.use.system.libpath = true oozie.wf.rerun.failnodes = true # Keytab specifics keyTabName = A.keytab keyTabUsername = A focusNodeLoginIng = A focusNodeLogin = A # Email notification list emailList = B
xml file <workflow-app xmlns="uri:oozie:workflow:0.4" name="bxdb"> <global> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> </global> <credentials> <credential name="hive2_credentials" type="hive2"> <property> <name>hive2.jdbc.url</name> <value>jdbc:hive2://${hive2_server}:${hive2_port}/default</value> </property> <property> <name>hive2.server.principal</name> <value>hive/${hive2_server}@${kerberos_realm}</value> </property> </credential> </credentials> <start to="sshFileTransfer"/> <action name="sshFileTransfer"> <ssh xmlns="uri:oozie:ssh-action:0.1"> <host>${focusNodeLoginIng}</host> <!-- Change the name of the script --> <command>/A/B/EsdToHDFS.sh</command> <args>A</args> <args> B</args> <args> C</args> <capture-output /> </ssh> <ok to="process-bxdb"/> <error to="sendEmailDQ_SRC"/> </action> <!-- Move from landing zone on HDFS to processing --> <!-- Emit whether data is complete or partial, together with timestamp --> <!-- Spark job to process the snapshots and cdr data --> <action name="process-bxdb"> <spark xmlns="uri:oozie:spark-action:0.2"> <master>yarn</master> <mode>cluster</mode> <name>Process BXDB</name> <class>IngestBXDB</class> <jar>bxdb_sproc_cataloguereport-1.0-SNAPSHOT.jar</jar> <spark-opts>--num-executors 8 --executor-cores 2 --executor-memory 4G --driver-memory 4g --driver-cores 2</spark-opts> <arg>${nameNode}/user/hive/warehouse/belltv_lnd.db/bxdb_sproc_cataloguereport</arg> <arg>Hello</arg> <arg>World</arg> </spark> <ok to="impala-refresh-iis"/> <error to="sendEmailDQ_SRC"/> </action> <!-- Impala invalidate/refresh metadata --> <action name="impala-refresh-iis"> <shell xmlns="uri:oozie:shell-action:0.3"> <exec>impala-command.sh</exec> <argument>${keyTabName}</argument> <argument>${keyTabUsername}</argument> <argument>${impala_server}</argument> <argument>refresh belltv_expl.bxdb_sproc_cataloguereport</argument> <file>${nameNode}/${keyTabLocation}/${keyTabName}</file> </shell> <ok to="end"/> <error to="fail"/> </action> <action name="sendEmailDQ_SRC"> <email xmlns="uri:oozie:email-action:0.1"> <to>${emailList}</to> <subject>Error in the workflow please verify</subject> <body>BXDB project returned an error please verify</body> </email> <ok to="fail"/> <error to="fail"/> </action> <kill name="fail"> <message>"BXDB ingestion failure"</message> </kill> <end name='end'/> </workflow-app>
command to run
oozie job -abc.properties -run
Created 04-03-2017 04:51 PM
Try addig some arguments into your Oozie run command like so:
$ oozie job -oozie http://localhost:11000/oozie -config job.properties -run
If those changes don't work for you might try the following: