Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

oozie coordinators s3n path problem

oozie coordinators s3n path problem

Explorer

Hello guys,

 

               I have installed the cloudera enterprise and  through HUE UI browser I can able to run my oozie workflow job my  input path as s3n://xxx-xxx/{YEAR}${MONTH}${DAY}/${HOUR} and output path (hdfs://...).When I coming to coordinators jobs as specfied the input

path through data-set (uri -template) as

 

s3n://xxx-xxx/{YEAR}${MONTH}${DAY}/${HOUR} its  preappend as

 

${nameNode}/user/${wf:user()}/s3n://ymxdata-test/${YEAR}${MONTH}${DAY}/${HOUR} like this.

 

help needed.This configuration through HUE only.. 

 

 

 

 

 

4 REPLIES 4

Re: oozie coordinators s3n path problem

Shouldn't the data be copied from S3 to HDFS anyway?

 

(at least at the Oozie level: https://issues.apache.org/jira/browse/OOZIE-426)

Re: oozie coordinators s3n path problem

Contributor

Yes, The data dependencies in a coordinator must be on HDFS.  However, the next version of Oozie will also allow it to be in HCatalog.

Software Engineer | Cloudera, Inc. | http://cloudera.com

Re: oozie coordinators s3n path problem

Explorer

(HUE Browser) But I can able to configure my  workflow job  input path as  s3n:// .why  can't be able to  configure the coordinator job dataset as s3n:// input path.

Highlighted

Re: oozie coordinators s3n path problem

Master Guru

Possible dupe thread: http://community.cloudera.com/t5/Cloudera-Manager-Installation/Problem-in-oozie-input-path-How-do-I-...

 

Your issue is in your configuration. You need to remove the templated "${nameNode}/user/${wf:user()}/" in your workflow.xml if you are already passing a proper URI.