Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cannot programmatically access S3N data in CDH 5.0.0/5.0.1

Cannot programmatically access S3N data in CDH 5.0.0/5.0.1

An app that I'm writing needs to copy data from an S3N file system to HDFS. However, when I attempt to do this using the hadoop .jar files provided with CDH 5.0.x, I get the following error:


java.lang.VerifyError: Bad type on operand stack
Exception Details:
org/apache/hadoop/fs/s3native/Jets3tNativeFileSystemStore.copy(Ljava/lang/String;Ljava/lang/String;)V @142: invokevirtual
Type 'org/jets3t/service/model/S3Object' (current frame, stack[4]) is not assignable to 'org/jets3t/service/model/StorageObject'
Current Frame:
bci: @142
flags: { }
locals: { 'org/apache/hadoop/fs/s3native/Jets3tNativeFileSystemStore', 'java/lang/String', 'java/lang/String' }
stack: { 'org/jets3t/service/S3Service', 'java/lang/String', 'java/lang/String', 'java/lang/String', 'org/jets3t/service/model/S3Object', integer }


I've uploaded the output from a Scala interpreter (with the Hadoop conf and lib folders on the path) illustrating the problem here. I've confirmed this on both the CDH 5.0 Virtualbox image and a CDH 5.0.1 that I manually created. Does the version of Hadoop that ships with CDH contain an incompatible version of jets3n? If so, are there any workarounds? 







Re: Cannot programmatically access S3N data in CDH 5.0.0/5.0.1

Master Guru
This seems like a problem stemming from something Scala does, from what am seeing.

When I attempt to build the FS via a S3N URI in a regular Java app, the exception is not seen although the construction trace follows the same path as the reported error stack trace in your paste.

Perhaps there's something incorrect, or a bug, within Scala that may be causing this and thats where you need to look?