Member since
01-11-2017
9
Posts
2
Kudos Received
0
Solutions
04-26-2018
06:27 PM
2 Kudos
We have a cluster on Azure with ADLS as a storage layer. I have updated core-site.xml with the keys so that I can access ADLS from hdfs cli. However, when I try to create an external table on hive using beeline from a csv file that is there on ADLS folder, I see a permission error which indicates that the end user doesn't have permission to read that csv file. I also checked to make sure hive impersonation is set to true that means it would be the hive user which needs to have access to the file but not the end user. But in this case, I am not sure why i see an error message about the end user not having permissions to read that file on ADLS. Can some shed some light on this and point me in the right direction? We have the cluster running with HDP 2.6.4 on Azure. Here is the SQL that i am using to create an external table ( i connected to hive via beeline as user 'ravi' ) CREATE EXTERNAL TABLE IF NOT EXISTS Cars( Name STRING, Miles_per_Gallon INT, Cylinders INT, Displacement INT, Horsepower INT, Weight_in_lbs INT, Acceleration DECIMAL, Year DATE, Origin CHAR(1)) COMMENT 'Data about cars from a public database' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE location 'adl://hdpadls.azuredatalakestore.net/folder1'; Error Message: Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [ravi] does not have [ALL] privilege on [adl://hdpadls.azuredatalakestore.net/folder1] (state=42000,code=40000)
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger
04-04-2018
07:31 PM
@vperiasamy Thanks. A follow-up question on HDFS RBACs via Ranger if the data is stored on one of the cloud-stroage platforms like ADLS/WASB/S3A etc... my research shows that we can't control access to the files via ranger. Can you please share your thoughts on this as well?
... View more
03-13-2018
09:54 PM
to be specific .. I would like to have ranger policies that can control Hive database, table, column level and also row-level policy when the data is stored on ADLS or WASB.
... View more
03-13-2018
09:47 PM
Yes. I am interested in RBAC on Hive Tables when HDI cluster is using ADLS/WASB for storage. Is that possible?
... View more
03-13-2018
05:02 PM
As per the following link, it is nice to see Ranger version 0.7.0 is supported with HDI 3.6. However, I would like to know if all of the ranger functionality/features supported with the latest HDI regardless of the underline storage selected for the cluster? Can someone list out any Ranger limitations that we may run into if we choose to have HDI 3.6 with VHDs/WASB/ADLS? https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-component-versioning
... View more
Labels:
- Labels:
-
Apache Ranger