About sudarshan_shrid

sudarshan_shrid · ‎07-19-2018

@sseth I have downloaded the latest jar https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-latest-hadoop2.jar Tried creating the external table and its failing with following error: FAILED: HiveAccessControlException Permission denied: user [abcd] does not have [READ] privilege on [gs://hdp-opt1/forhive/languages] (state=42000,code=40000) I have enabled Hive plugin and set the permission of 777 in coresite.xml Where there any changes made to jar?? I also see few properties have changed in this link: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_cloud-data-access/content/gcp-cluster-config.html Is it mandatory to use the json key? If my vm instance has required permission to talk to gcs

sudarshan_shrid · ‎06-08-2018

Hi @sseth Issue is resolved after adding following property in core-site.xml fs.gs.reported.permissions=777 Normal Users can access hive and create external table pointing to GCS location.

sudarshan_shrid · ‎05-31-2018

@Geoffrey Shelton Okot I was able to create hive external table pointing my storage as GCS. But it only works as hive superuser but doesn't work as a normal hive user meaning, hdpuser1 cannot create hive table it fails with above error, but if execute su - hive it works . I am no sure how to rectify this.

sudarshan_shrid · ‎05-29-2018

@Geoffrey Shelton Okot I did try, but still fails. CommandException: hdpuser1:WRITE is not a valid ACL changehdpuser1 is not a valid scope type The GCS bucket has storage admin rights given to service account hadoop fs -ls gs://bucket/ = works fine

sudarshan_shrid · ‎05-29-2018

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.security.AccessControlException: Permission denied: user=hdpuser1, path="gs://bucket/":hive:hive:drwx------) (state=08S01,code=1) hdpuser1 is an AD user, using the same user I execute $ hdfs dfs -ls gs://bucket/ but using beeline when I try to create an external table it fails

sudarshan_shrid · ‎05-28-2018

Hi All, In the cloud world, can I have mutilple HDP VM instances use a single ranger DB, wherein my Ranger DB is sitting outside the HDP VM instance and i am migrating my existing ranger policies to MYSQL DB

sudarshan_shrid · ‎05-28-2018

Hi Neeraj, We are trying to test GCS connector with HDP 2.6.5 ( https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_cloud-data-access/content/authentication-gcp.html ) by having GCS as my storage. When trying to create hive external table it’s failing with the following error: Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.security.AccessControlException: Permission denied: user=hdpuser1, path="gs://hive_metastore/":hive:hive:drwx------) (state=08S01,code=1) Syntax: CREATE EXTERNAL TABLE test1256(name string,id int) LOCATION 'gs://hive_metastore/' ;

sudarshan_shrid · ‎05-28-2018

Hi, Thanks a lot for the info, but still facing the same issue. I did create the user in AD and have a valid ticket , hdfs command does work accessing the GCS but cannot create external hive table.

sudarshan_shrid · ‎05-25-2018

I have installed a hadoop 2.6.5 version cluster in GCP using VM's instances. Used GCP connector and pointed by hdfs to use gs bucket. Added the below 2 entries in coresite.xml : google.cloud.auth.service.account.json.keyfile=<Path-to-the-JSON-file> fs.gs.working.dir=/ When using hadoop gs -ls / works fine , but when I am creating a hive tables CREATE EXTERNAL TABLE test1256(name string,id int) LOCATION 'gs://bucket/'; I get the following error: Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.security.AccessControlException: Permission denied: user=hdpuser1, path="gs://bucket/":hive:hive:drwx------) (state=08S01,code=1) Apart form changes to coresite.xml are there any changes to be made at hive.xml also? https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_cloud-data-access/content/authentication-gcp.html

sudarshan_shrid · ‎08-17-2017

Hi, what's the maximum number of policies retrieved via curl -iv -u admin:admin -H "Content-type:application/json" -X GET http://xxxx:6080/service/public/api/policy i get max 200 rules for hive i know for sure thee are more rules..how to get that using ranger api

Online	Offline
Last Visited	‎08-15-2018 02:36 AM

Member Since	‎08-17-2017 05:56 PM
Last Visited	‎08-15-2018 02:36 AM
Posts	36
Kudos received	1

Cloudera Community

Re: Hive with Google Cloud Storage

Re: Hive with Google Cloud Storage

Re: Hive with Google Cloud Storage

Re: Hive with Google Cloud Storage

Re: Hive with Google Cloud Storage

Can mutilple HDP VM instances use a single ranger...

Re: Hive and Google Cloud Storage

Re: Hive with Google Cloud Storage

Hive with Google Cloud Storage

Re: Ranger Policies using API