Reply
New Contributor
Posts: 1
Registered: ‎09-25-2013

Problems with Hive and S3 .

I am having issues accessing S3 with hive.  I have my keys and ID under hive-site.xml and i am able to see them when I run

hive> set;  

 

I get a list of all the attributes for hive but when I try to access s3 to create a table i get the following error.

 

FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively).)

 

I even set the fs.s3 and fs.s3n attributes manually using the set command under hive. 

 

Still no luck .  Any suggestions or ideas

 

Thanks so much

Posts: 1,508
Kudos: 260
Solutions: 230
Registered: ‎07-31-2013

Re: Problems with Hive and S3 .

Please try placing the properties in your Hadoop config's core-site.xml.

If you use Cloudera Manager, add the fs.s3.* and fs.s3n.* properties in XML form to CM -> HDFS -> Configuration -> "core-site.xml cluster wide safety valve" and save plus do a CM -> Actions -> Deploy Client Configuration.
Backline Customer Operations Engineer
New Contributor
Posts: 1
Registered: ‎09-12-2014

Re: Problems with Hive and S3 .

Do you copy the entire block of XML into the Value text box? Like:

 

<property>
<name>fs.s3.awsAccessKeyId</name>
<value>asdfasd</value>
</property>

<property>
<name>fs.s3.awsSecretAccessKey</name>
<value>asdfasdf</value>
</property>

 

I did that, deployed the client configuration, and then went to /etc/hadoop/conf/core-site.xml and there were no changes. STill can't see S3 buckets.

New Contributor
Posts: 1
Registered: ‎07-26-2017

Re: Problems with Hive and S3 .

Hi,

Do you able to resolve this issue. I understand that its been a long time, since you have posted this. Now I am facing the same Issue, your suggession for resolving this would be helpful for me.

 

Thanks in advance.

Announcements