Problem in oozie input path . How do I configure the oozie in cloudera manager,input path as s3n.

[#1]

My input path in s3n,

s3n://xxx-xxx/20130813/08

My oozie configuration show as ,

hdfs://xxx.internal:8020/s3n://xxx-xxx/20130813/08

[#2]

Yes, it should work, cf. above users using it with no problem. What is your Oozie version?

[#3]

I'm using "oozie-3.3.2-cdh4.5.0". The installation is working fine when input/output paths are specified in form "hdfs://…." but when specified with either "s3://…" or "s3n://" we get the following error in the logs:

Scheme of ʼs3n://...ʼ is not supported.

Where "…" equals path to input/output, verified there, verified working with hadoop commands in console.

Anything you know of I can look at?

[#4]

Nevermind, I got it.

The solution was a mix of upgrading the version (to previously listed version) AND adding the supported filesystems property back in.

thanks!

[#5]

Glad to hear!

[#6]

---

@Ashok wrote:

My input path in s3n,

s3n://xxx-xxx/20130813/08

My oozie configuration show as ,

hdfs://xxx.internal:8020/s3n://xxx-xxx/20130813/08

---

Can you share your workflow.xml for us to validate?

If you're passing an S3 input or output path, simply ensure your workflow does not template it as ${nameNode}/${input} or something like that. That way you're prepending a HDFS URI to your already-an-uri path. This could most likely be your issue.

[#7]

In coordinator jobs i'm passing the dataset  uri template as 

s3n://xxx-xxx/${YEAR}${MONTH}${DAY}/${HOUR}

and coord:dataOut as 

<property>
<name>in_folder</name>
<value>${coord:dataOut('in_folder')}</value>
</property>

and my workflow.xml  input as 

${in_folder}

when I submit the  coordinator job it automatically preappend  the configuration like:

${nameNode}s3n://xxx-xxx/${YEAR}${MONTH}${DAY}/${HOUR}

[#8]

Good to know, Hue Coodinators are prepended only with hdfs.

Is https://issues.apache.org/jira/browse/OOZIE-426 finished?

[#9]

FWIW, the same job works fine as a workflow when submitted via Hue. In this case, we manually pass the input (S3) and output (hdfs) locations and the job runs successfuly - thus establishing that the problem is not with S3 support. The problem is when we let the co-ordinator pass this input (via a computed datasource) does it automatically prepend hdfs://{nameNode} in front of the s3n://<> URI. Hope this clarifies.

[#10]

Ok this clarifies a lot! I updated https://issues.cloudera.org/browse/HUE-1501.

[#11]

Thanks. Is this considered as a bug? If yes, what are some workarounds that we can follow for now? Any help is appreciated.