Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

SOLVED Go to solution
Highlighted

Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

Contributor

Hi All,

I added  fs.s3n.awsAccessKeyId and fs.s3n.awsSecretAccessKey properties in core,hdfs,hive-site.xml. I am able to run select * from tbl which is on s3 and got the resullt with beeline.But, when i run select count(*) from tbl, it's getting failed with following errors

Error: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
       ........
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:251)
        ... 11 more
Caused by: java.io.IOException: s3n://rakeshs3 : 400 : Bad Request
        at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:453)
        at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:427)
                 ... 16 more
Caused by: org.jets3t.service.impl.rest.HttpException: 400 Bad Request
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:425)
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279)
              ... 29 more

 

Impala Errors:

Failed to open HDFS file s3n://rakeshs3/tel.txt    Error(255): Unknown error 255

 

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

Contributor

 

 

MR/Hive Jobs Issue resolved by replacing old jars with new version and switched to s3a. Below are the jars that are replaced.

jets3t jar

aws-java-sdk jars

jackson jars

4 REPLIES 4

Re: Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

Contributor

Hello Rakesh,

 

Unfortunately s3n filesystem scheme is not supported by Impala: http://www.cloudera.com/documentation/enterprise/latest/topics/impala_s3.html#s3_restrictions

 

If you set up s3a, please look at the recommended settings at "Best Practices for Using Impala with S3" section on the same page.

 

Regards

Miklos Szurap

Customer Operations Engineer

Re: Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

Contributor
Thanks Miklos..

What about hive MR jobs ?

Re: Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

Contributor

Hi,

 

If you see the same for Hive/MapReduce jobs, then please can you confirm that the table in question has many partitions? We have seen similar problems for tables where it contained over thousand or ten thousands of partitions.

Re: Hive jobs are not running on s3n/impala failed to open hdfs files s3n://my_bucket

Contributor

 

 

MR/Hive Jobs Issue resolved by replacing old jars with new version and switched to s3a. Below are the jars that are replaced.

jets3t jar

aws-java-sdk jars

jackson jars