About yagoaparecidoti

yagoaparecidoti · ‎09-29-2022

hello cloudera community, we are having problems using livy with job spark to read hive by jupyter notebook when we run a simple query, for example: "spark.sql("show show databases").show()" returns the error below "org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient" Could you help us with this setup? ps: we are using cdh 5.16.x

yagoaparecidoti · ‎09-21-2022

hello cloudera community, we are having a problem accessing the hive cli of a certain user and the spark-shell too when executing the query "show databases" in hive cli returns the error: "FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset " when executing the "show databases" query in spark-shell returns the error: "org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException;" "WARN metastore.RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect. org.apache.thrift.transport.TTransportException" when we use beeline and run the query "show databases" it works without problem could you help us with this problem? we are using cloudera manager 5.16.1 and cdh 5.16.1 the cluster is with kerberos and sentry is managing the permissions to the cluster databases

yagoaparecidoti · ‎09-15-2022

hi @araujo the userPrincipalName of user livy is: livy/hostname_livy_server@DOMAIN.LOCAL the userPrincipalName of the livy-http user is: livy-http@DOMAIN.LOCAL running the command "kinit livy": running the command "kinit livy-http": running the "kinit" command with the keytab created for user livy: running the command "kinit" with the keytab created for the user livy-http: we've been facing this problem for months, we haven't found the solution yet.

yagoaparecidoti · ‎09-12-2022

hi @Scharan the same goes for sentry?

yagoaparecidoti · ‎09-12-2022

hi @Scharan so, does the ranger only work the permissions if it uses beeline or connections that access the hiveserver2 address?

yagoaparecidoti · ‎09-12-2022

we are using ambari 2.6.2.2 with hdp 2.6.5 the permissions are being managed by the ranger, but checking the ranger settings, it's only managing the /apps/hive/warehouse directory on the hdfs the problem we are having is in the /area/data/pll directory on the hdfs, this directory has acl rwx permission for the user using beeline to create an external table is returning the error: Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [service_user] does not have [ALL] privilege on [hdfs://service1/area/data/pll] (state=42000,code=40000) but using hive cli does not return error, table is created successfully. What could be causing this problem?

yagoaparecidoti · ‎08-30-2022

hi @araujo yes, we know the passwords, because we created these two users from scratch before creating the keytabs for the two users, we managed to kinit the two users without problem "kinit user", after creating the keytabs for the two users, kinit only works with the keytab, but it only works on the livy user, when we try to run kinit in livy-http user keytab displays the error "kinit: Preauthentication failed while getting initial credentials" the userprincipalname of each user is: livy: livy/hostname_livy_server@DOMAIN.LOCAL livy-http: HTTP/hostname_livy_server@DOMAIN.LOCAL

yagoaparecidoti · ‎08-30-2022

hi @araujo this command "SHOW ROLE GRANT GROUP group name" I know and already used, I thought there was a command that did the opposite. well I will check in mysql then and soon return if it worked.

yagoaparecidoti · ‎08-29-2022

hello cloudera community, how can we find out which group is tied to a sentry role by command line?

yagoaparecidoti · ‎08-29-2022

hello cloudera community, could you help us with a problem regarding the /var/log directory at certain times (we couldn't identify the right moment) the /var/log directory is changing the permissions to 750 and with that the CDH 6.3.x cluster services is failing because it can't write the logs. Have you ever had a problem like this? did you manage to solve it? we had to leave a crontab running every 5 minutes to always change it to 755. ps: selinux is disabled ps: firewall is disabled

Online	Offline
Last Visited	‎01-09-2025 02:12 PM

Member Since	‎02-28-2022 11:49 AM
Last Visited	‎01-09-2025 02:12 PM
Posts	147
Kudos received	13

Cloudera Community

Re: how to create a flow in nifi that reads a hive...

Re: how create, export and import full nifi templa...

Re: [Cloudera][ImpalaJDBCDriver](500593) Communica...

Re: deploy rack concept in cluster

Re: yarn run logs - error getting logs at hostname...

error in livy - org.apache.hadoop.hive.ql.metadata...

error executing queries in hive cli and spark-shel...

Re: problem generating keytab with HTTP SPN

Re: permission failure when using beeline

Re: permission failure when using beeline

permission failure when using beeline

Re: problem generating keytab with HTTP SPN

Re: how to know which group is linked to a sentry ...

how to know which group is linked to a sentry role...

/var/log directory is being changed permission to ...