New Contributor
Posts: 2
Registered: ‎10-09-2014

Unable to write to HDFS from Oozie shell action

I am invoking the below Oozie shell action to invoke a shell script which writes an output to a file. I have tried this on Cloudera 5.7 VM ...Whenever I am trying to write to a new file in HDFS the job fails.

I checked the application/job id logs but had no info. Is there any restriction to write to hdfs from a shell script?


In the shell script below, I am trying to query on a table and write the results to a file.

hive -e "create t1(id int); insert into t1 values(1);"
hive -e "select * from t1;" > /user/cloudera/test123/t1_data.txt


<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.5" name="shell.workflow">
<start to="shell-node"/>
<action name="shell-node">
<shell xmlns="uri:oozie:shell-action:0.1">
<delete path="test123"/>
<mkdir path="test123"/>

<ok to="end"/>
<error to="kill"/>
<kill name="kill">
<message>Error in Shell.Please refer the Oozie Logs</message>
<end name="end"/>




Posts: 177
Topics: 8
Kudos: 28
Solutions: 19
Registered: ‎07-16-2015

Re: Unable to write to HDFS from Oozie shell action



First of all, the following command will not write to HDFS but to the local FS of the node executing the task.

"hive -e "select * from t1;" > /user/cloudera/test123/t1_data.txt"


Does this path exist ? And does the user runing the shell have write permission on that path (on the local FS).


Our community is getting a little larger. And a lot better.

Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.