Created 12-16-2016 02:19 AM
Unable to create table pointing to S3 after enabling Ranger.
This is database we created before enabling Ranger.
SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem; SET fs.s3a.access.key=xxxxxxx; SET fs.s3a.secret.key=yyyyyyyyyyyyyyy; CREATE DATABASE IF NOT EXISTS backup_s3a1 COMMENT "s3a schema test" LOCATION "s3a://gd-de-dp-db-hcat-backup-schema/";
After Ranger was enabled, we try to create another database but it is throwing error.
0: jdbc:hive2://usw2dxdpmn01.local:> SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem; Error: Error while processing statement: Cannot modify fs.s3a.impl at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)
How do I whitelist the fs.s3* parameters in Ranger ?
Created 12-20-2016 02:30 AM
I resolved the problem by adding this configuration in custom-hiveserver2-site.xml
hive.security.authorization.sqlstd.confwhitelist.append=fs\.s3a\..*|fs\.s3n\..* |
Created 08-11-2017 03:18 PM
S3 Seems to be working fine with HDP 2.6 as You have the option to Define URL in Ranger Hive Policy Section
Created 08-11-2017 03:31 PM
Created 08-11-2017 06:17 PM
This is a new option available as TP in HDP 2.6.