The exception is odd if it presents itself upon a simple fs.exists(new Path("/d/w/3/2/Hourly/optimized_preview")) call, but how certain are you on if it is that for sure? I do not see that call in the stack trace but perhaps it is chopped off.   The exception can be expected if they do call fs.exists(…) with path components that instead treat optimized_preview as a directory, which am rather certain they are or the tool is doing, cause the ACE inode field only shows upto the last parent, and thats quoting the whole file. For example, fs.exists(new Path("/d/w/3/2/Hourly/optimized_preview/file")). ... View more

These are available in the base version of Apache Hive shipped with CDH5 (beta currently). The CDH4 equivalent is on a stable release launched before the features were added upstream. You could use Justin's guide to get a custom build of newer version of Hive running on your CDH4 cluster - this is possible to do. ... View more

You're right that you'll need to build your own failover on the client side for WebHDFS as it presently lacks HA awareness and support. Another easier alternative is to setup and use HttpFs as the REST gateway, which is HA-aware and offers the exact same WebHDFS API and functionality. ... View more

You could do (a) with the SpoolingDirectory source, as it allows for the event to carry the original filename (via a custom sink wrapper that looks for it) but doing (b) doesn't fit in with the event delivery mechanism of Flume and AFAICT, its not possible to do directly. ... View more

Those entries in core-site.xml are the solution to the impersonation error, but you'll need to ensure you've made the change on the NameNode and JobTracker's core-site.xml and that you've restarted them after the change. Has this been done as well? P.s. If you use Cloudera Manager controlled cluster, this property is pre-added for the out of box experience. ... View more

@dvohra wrote: This isn't true. Depending on what you're doing with Oozie, S3 is supported just fine as an input or output location.   Doesn't the coordinator expect the input path to be on HDFS as hdfs://{nameNode} is prepended automatically? The workflow.xml is on the HDFS? Isn't the workflow.xml required to be on the HDFS? Yes unfortunately coordinators currently poll inputs over HDFS alone, which is a limitation. However, writing simple WF actions to work over S3 is still possible.   Yes, WFs should reside on HDFS, as Oozie views it as its central DFS. Similar to how MR requires a proper DFS to run. But this shouldn't impair simple I/O operations done over an external FS such as S3.   I think Romain has covered the relevant JIRAs for tracking removal of this limitation. ... View more

You will need to add your hive-contrib-*.jar under $HIVE_HOME/lib/ as an "add jar" command before you can execute this, as the class seems to be from an off-default contrib package. ... View more

Do you mean the administrative documentation at pages such as http://www.cloudera.com/content/support/en/documentation/cdh4-documentation/cdh4-documentation-v4-latest.html? If so, you should be able to see a small PDF link next to any major topic to be able to grab it in PDF form.   Here's the CDH4 Installation Guide as a PDF for example: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/PDF/CDH4-Installation-Guide.pdf ... View more