Member since
04-03-2019
962
Posts
1743
Kudos Received
146
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
11371 | 03-08-2019 06:33 PM | |
4845 | 02-15-2019 08:47 PM | |
4146 | 09-26-2018 06:02 PM | |
10525 | 09-07-2018 10:33 PM | |
5580 | 04-25-2018 01:55 AM |
04-04-2016
06:31 PM
@Aaron Dossett - Can you please check below comment by Artem
... View more
04-04-2016
01:15 PM
3 Kudos
@Aaron Dossett Please don't remove these JARs 🙂 Yes its by design to support more functionality and fix existing bugs.
... View more
04-04-2016
08:56 AM
No problem! @Rajendra Vechalapu
... View more
04-04-2016
07:33 AM
4 Kudos
@Rajendra Vechalapu You can use hdfs commands or use hue to access hdfs file browser username - hue password - 1111 OR You can login to Ambari server and use file browser view URL - http://127.0.0.1:8080 username - admin password - admin
... View more
04-04-2016
05:06 AM
1 Kudo
@grajagopal - In addition to all the answers, you can also include hive-site.xml in workflow using below tag <job-xml>$path/hive-site.xml</job-xml> Note - You need to put above tag after </prepare> tag. If you don't have <prepare> tags then you can put this after <namenode>. Please note that order is very important here or else you will get XML schema error.
... View more
04-03-2016
05:17 PM
@allen huang - If you know the application id then you can get logs via command line which would give you some clue. yarn logs -applicationId <application-id>
... View more
04-03-2016
07:50 AM
2 Kudos
You can do what @Arti Wadhwani said or you can add hbase-site.xml under <job-xml> in your oozie workflow.xml Below is the example snippet: <start to="hive-node"/>
<action name="hive-node" cred="my_auth,hbaseauth">
<hive xmlns="uri:oozie:hive-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/hive"/>
<mkdir path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data"/>
</prepare>
<job-xml>hbase-site.xml</job-xml>
... View more
04-02-2016
09:23 AM
1 Kudo
@hoda moradi Can you please check if permissions for /tmp directory on hdfs is set to 777 and owner and group set to hdfs? Also, Please double check permissions for /mr-history/ it should be set to 777 and owned by mapred followed by hadoop for group permissions.
... View more
04-01-2016
04:23 PM
11 Kudos
This articles explains about how to re-run only failed action for oozie workflow. Below are the steps: 1. Find out the WF id of the failed/killed job. 2. Prepare a job config file which needs to be passed to the rerun command. To do so Follow below steps: 2.1 You first need the oozie job configuration xml file. The easiest way to do that is to use the -configcontent option of the oozie job command. E.g. On commandline export OOZIE_URL="http://<oozie-host>:11000/oozie"
oozie job -configcontent <workflow-id> > job_conf.xml 2.2 Delete oozie.coord.application.path property from job_conf.xml. This is to avoid E0301: Invalid resource oozie rerun error. 2.3 Now add below property in job_conf.xml. This determines what actions need to be run in the workflow. If we specify specific action nodes here then it will skip those actions. if nothing specified then it will run all actions of the workflow.
To run all actions of a workflow: <property>
<name>oozie.wf.rerun.skip.nodes</name>
<value>,</value>
</property>
To skip few actions of a workflow ( all the action nodes specified here will be skipped and the rest will be run 😞 <property>
<name>oozie.wf.rerun.skip.nodes</name>
<value>action-name1,action-name2,etc.</value>
</property> 3. Re-run wf with below command oozie job -config "job_conf.xml" -rerun <wf-id>
... View more
Labels:
04-01-2016
02:59 PM
4 Kudos
@mike pal - Good question! For case number 1
1. Find out the WF id of the failed/killed job. 2. Prepare
a job config file which needs to be passed to the rerun command. To do so
Follow below steps: 2.1 You first need the oozie job
configuration xml file. The easiest way to do that is to use the
-configcontent option of the oozie job command. E.g. On
commandline export OOZIE_URL="http://<oozie-host>:11000/oozie"
oozie job -configcontent <workflow-id> > job_conf.xml 2.2 Delete
oozie.coord.application.path property from job_conf.xml. This is to avoid E0301: Invalid
resource oozie rerun error. 2.3 Now
add below property in job_conf.xml. This determines what
actions need to be run in the workflow. If we specify specific action nodes
here then it will skip those actions. if nothing specified then it will run all
actions of the workflow. To run all actions of a workflow: <property>
<name>oozie.wf.rerun.skip.nodes</name>
<value>,</value>
</property> To skip
few actions of a workflow ( all the action nodes specified here will be
skipped and the rest will be run 😞 <property>
<name>oozie.wf.rerun.skip.nodes</name>
<value>action-name1,action-name2,etc.</value>
</property> 3. Re-run wf with below command oozie job -config "job_conf.xml" -rerun <wf-id>
... View more