Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Solved Go to solution

Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Explorer

After I upgrade from CDH 5.3.1 to CDH 5.3.2, and then run "select" SQL queries, it gives me some errors:

 

select * from my_table limit 10;

FAILED: SemanticException Unable to determine if s3n://my-bucket/is encrypted: java.lang.IllegalArgumentException: Wrong FS: s3n://my-bucket/, expected: hdfs://us0701aohc172018080109.tangome.gbl:8020

 

 

I tryied to drop my table and re-create it again add partitions to it, then run select queries, still the same error. Before upgrading everything is OK.

 

Does anybody happen to know why?

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Cloudera Employee
Hi,

You just hit a bug related with the encryption changes added to CDH 5.3.2. This happens because an internal function is trying to check if the table location is encrypted or not. This works only for HDFS files, but not for external locations.

I'm afraid the only way to make it work is to downgrade to CDH 5.3.1. I don't see a workaround to bypass this error.

The next CDH 5.4.0 version will include this bugfix.

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Cloudera Employee
Hi,

This issue has been fixed on 5.3.3 and 5.4.0.

10 REPLIES 10

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Cloudera Employee
Hi,

You just hit a bug related with the encryption changes added to CDH 5.3.2. This happens because an internal function is trying to check if the table location is encrypted or not. This works only for HDFS files, but not for external locations.

I'm afraid the only way to make it work is to downgrade to CDH 5.3.1. I don't see a workaround to bypass this error.

The next CDH 5.4.0 version will include this bugfix.

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Explorer

Thank you for telling me this, you really save my day!

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Cloudera Employee
You're welcome. Btw, we're expecting to have this fix for 5.3.3 instead.

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

New Contributor

How to downgrade?

By following the http://www.cloudera.com/content/cloudera/en/documentation/core/v5-2-x/topics/cdh_ig_install_previous...

 

It does not work for me. I am using ubuntu trusty. 

Although I specified the 5.3.1 version, apt-get install hive failure. 

If I do not specified the version, it will install 5.3.2

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

New Contributor

have the same issue on an external table in CDH 5.3.3

Error while compiling statement: FAILED: SemanticException Unable to determine if hdfs://cdhtest/store/TPCH3TB/lineitem is encrypted: java.lang.IllegalArgumentException: Wrong FS: hdfs://cdhtest/store/TPCH3TB/lineitem, expected: hdfs://pupa

 

any workaround ? is it fixed in 5.4

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Cloudera Employee
Hi,

This issue has been fixed on 5.3.3 and 5.4.0.

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Explorer

Thanks, I'll upgrade my cluster and try it again

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

New Contributor

This is still not working for me. I am currently using cdh-5.4.3 for hive and impala

Highlighted

Re: Can NOT select from external tables on S3 after upgrading to CDH 5.3.2

Explorer

Despite the above comments about 5.3.3 and 5.4.0, this issue is not fixed in those versions. According to the comments from Jim B on a related thread of mine ( http://community.cloudera.com/t5/Batch-SQL-Apache-Hive/Loading-to-S3-Fails-CDH-5-3-0/m-p/24088#U2408... ) this _IS_ fixed in 5.4.4