Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 12-16-2016 02:19 AM
Unable to create table pointing to S3 after enabling Ranger.
This is database we created before enabling Ranger.
SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem; SET fs.s3a.access.key=xxxxxxx; SET fs.s3a.secret.key=yyyyyyyyyyyyyyy; CREATE DATABASE IF NOT EXISTS backup_s3a1 COMMENT "s3a schema test" LOCATION "s3a://gd-de-dp-db-hcat-backup-schema/";
After Ranger was enabled, we try to create another database but it is throwing error.
0: jdbc:hive2://usw2dxdpmn01.local:> SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem; Error: Error while processing statement: Cannot modify fs.s3a.impl at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)
How do I whitelist the fs.s3* parameters in Ranger ?
Created 12-20-2016 02:30 AM
I resolved the problem by adding this configuration in custom-hiveserver2-site.xml
hive.security.authorization.sqlstd.confwhitelist.append=fs\.s3a\..*|fs\.s3n\..* |
Created 08-11-2017 03:18 PM
S3 Seems to be working fine with HDP 2.6 as You have the option to Define URL in Ranger Hive Policy Section
Created 08-11-2017 03:31 PM
Created 08-11-2017 06:17 PM
This is a new option available as TP in HDP 2.6.