Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Ranger and creating database pointing to S3 not working

avatar
Expert Contributor

Unable to create table pointing to S3 after enabling Ranger.

This is database we created before enabling Ranger.

SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem;
SET fs.s3a.access.key=xxxxxxx;
SET fs.s3a.secret.key=yyyyyyyyyyyyyyy;


CREATE DATABASE IF NOT EXISTS backup_s3a1
COMMENT "s3a schema test"
LOCATION "s3a://gd-de-dp-db-hcat-backup-schema/";

After Ranger was enabled, we try to create another database but it is throwing error.

0: jdbc:hive2://usw2dxdpmn01.local:> SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem;
Error: Error while processing statement: Cannot modify fs.s3a.impl at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)

How do I whitelist the fs.s3* parameters in Ranger ?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

I resolved the problem by adding this configuration in custom-hiveserver2-site.xml

hive.security.authorization.sqlstd.confwhitelist.append=fs\.s3a\..*|fs\.s3n\..* |

View solution in original post

12 REPLIES 12

avatar
Contributor

S3 Seems to be working fine with HDP 2.6 as You have the option to Define URL in Ranger Hive Policy Section

avatar

This is a new option available as TP in HDP 2.6.