Reply
Explorer
Posts: 18
Registered: ‎08-13-2013

oozie coordinators s3n path problem

[ Edited ]

Hello guys,

 

               I have installed the cloudera enterprise and  through HUE UI browser I can able to run my oozie workflow job my  input path as s3n://xxx-xxx/{YEAR}${MONTH}${DAY}/${HOUR} and output path (hdfs://...).When I coming to coordinators jobs as specfied the input

path through data-set (uri -template) as

 

s3n://xxx-xxx/{YEAR}${MONTH}${DAY}/${HOUR} its  preappend as

 

${nameNode}/user/${wf:user()}/s3n://ymxdata-test/${YEAR}${MONTH}${DAY}/${HOUR} like this.

 

help needed.This configuration through HUE only.. 

 

 

 

 

 

Cloudera Employee
Posts: 721
Registered: ‎07-30-2013

Re: oozie coordinators s3n path problem

Shouldn't the data be copied from S3 to HDFS anyway?

 

(at least at the Oozie level: https://issues.apache.org/jira/browse/OOZIE-426)

Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: oozie coordinators s3n path problem

Yes, The data dependencies in a coordinator must be on HDFS.  However, the next version of Oozie will also allow it to be in HCatalog.

Software Engineer | Cloudera, Inc. | http://cloudera.com
Explorer
Posts: 18
Registered: ‎08-13-2013

Re: oozie coordinators s3n path problem

(HUE Browser) But I can able to configure my  workflow job  input path as  s3n:// .why  can't be able to  configure the coordinator job dataset as s3n:// input path.

Highlighted
Posts: 1,836
Kudos: 416
Solutions: 295
Registered: ‎07-31-2013

Re: oozie coordinators s3n path problem

Possible dupe thread: http://community.cloudera.com/t5/Cloudera-Manager-Installation/Problem-in-oozie-input-path-How-do-I-...

 

Your issue is in your configuration. You need to remove the templated "${nameNode}/user/${wf:user()}/" in your workflow.xml if you are already passing a proper URI.

Announcements