About Shifu

Shifu · ‎08-15-2021

Hi @Nil_kharat If you are running a hive job and if you noticed it is slower than the earlier run then you may need to check few things in the below order. 1. Is there any change in the code, 2. Is there any new data load added to the table which is getting processed, 3. Beeline console output where you can see the Hive counters details, 4. Hiveserver2 and Hivemetastore logs to see the compilation and execution time of the faster and slower run. 5. QueryId of the job(to check in HS2 and application logs) 6. Faster and slower run of application logs where identify the slowness. 7. explain <query> for the faster and slower(to see the record change and stats info) If you are happy with the response mark it as Accept as Solution

Shifu · ‎08-14-2021

Hi @Nil_kharat Yes, you can also use the particular lsof command as well to figure out the number of established connections to the particular port.

Shifu · ‎08-13-2021

Hi @amitshanker Thanks for the update. I can see than you had set hive.server2.webui.use.spnego as true, that means kerberos is enabled in your cluster. If spnego and kerberos are enabled then few settings needs to be changed in browser. Could you please follow the below link. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_sg_browser_access_kerberos_protected_url.html

Shifu · ‎08-12-2021

Thanks for the details. Can you share the hive-site.xml file Can you share me the complete screenshot of the error you are facing Do you have kerberos enabled in your cluster?

Shifu · ‎08-12-2021

Hi @amitshanker Could you let me know whether you are using CDH/HDP/CDP? What is the document which you had followed to enable HS2 UI? Can you share me the screenshot of the error you are facing Do you have kerberos enabled in your cluster?

Shifu · ‎08-12-2021

Hi @ryu [root@test02 ~]# hdfs dfs -rmr /tmp/root/testdirectory ... ... 21/08/11 12:08:30 WARN fs.TrashPolicyDefault: Can't create trash directory: hdfs://test/user/root/.Trash/Current/tmp/root ... ... rmr: Failed to move to trash: hdfs://test/tmp/root/testdirectory: Permission denied: user=root, access=WRITE, inode="/user":hdfs:hdfs:drwxr-xr-x Upon checking the logs you are trying to delete a testdirectory and if you do that it will try to move the file to trash directory(since trash is enabled) under "/user/root/.Trash" location. Since the folder /user("inode="/user":hdfs:hdfs:drwxr-xr-x") has hdfs as primary and group so eventually the user root falls under others(that is the third one r-x). The other users does not have write permission so that he was not able to write. Either give write permission for user root in to the folder or try to delete the folder as hdfs user to overcome the issue. If you are happy with the reply, mark it Accept as Solution

Shifu · ‎08-10-2021

Hi @Nil_kharat Question: Can anyone tell me how to check how many users are connected to the hive server2? Answer: Login to the server where hiveserver2 is running and run the below command to figure out the number of connections to the HS2 # netstat -ntpl | grep 10000 In the above command 10000 is the HS2 port number You can also check this from Cloudera manager CM > Hive_on_Tez > Instances > HiveServer2 > Open Connections And along with can we see which user is connected? If you enable Hiveserver2 UI then under Active Sessions you can see which user connected from which IP. Please check the below link. https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-WebUIforHiveServer2 If you are happy with the answer, mark it Accept as solution

Shifu · ‎07-25-2021

Hi @michael_boulter Could you let us know what is HDP version you are using? There is no TEZ UI from HDP 3 and in CDP. Could you check whether the Hive is running in port 10000/10001? Could you cross-check you were able to log in to beeline and run random queries?

Shifu · ‎07-19-2021

I can see you are in HDP but you are using Cloudera Driver(https://www.cloudera.com/downloads/connectors/hive/odbc/2-6-4.html) The first thing you need to do is start using Hortonworks Driver if you are using Ambari, link to download and configure is present in https://www.cloudera.com/downloads/hdp.html How to configure it? Check out the below link https://docs.cloudera.com/HDPDocuments/other/connectors/hive-jdbc/2.6.7/Simba-Hive-JDBC-Install-and-Configuration-Guide.pdf The error which had been shared should not harm connecting to Hive. To figure out the exact reason for the ODBC failure you need to enable TRACE logs as below You can configure the log level following those ways: If the client uses an ODBC on MS Windows machines, set the Log level on the Driver to TRACE and try a connection attempt, gather the driver logs: ODBC Driver Configuration > DataSourceAdministrator> select the DSN> Configure > Logging Options > set LogLevel: TRACE ODBC Driver Configuration > DataSourceAdministrator> select the DSN> Configure > Logging Options > set Log Path: "C:/" OK > OK ODBC Driver Configuration > DataSourceAdministrator> select the DSN> Configure > Test...

Shifu · ‎07-11-2021

Hi @srinivasp I believe you are missing SELECT and LIST privilege for the table in ranger for the user. Can you try to list the tables using Hive user from beeline and check whether you are seeing the same problem?

Online	Offline
Last Visited	‎05-11-2022 05:47 AM

Member Since	‎03-29-2020 10:09 PM
Last Visited	‎05-11-2022 05:47 AM
Posts	110
Kudos received	9

Cloudera Community

Re: Hive table in power bi

Re: How do we identify hive metastore performance ...

Re: Halting due to Out Of Memory Error...Exit code...

Re: Error while executing hive merge query

Re: Upgrading Individual Components Post HDP 3.1.5

Re: YARN job is Running Slow

Re: How many users connected to HiveServer2

Re: Hive webUrl issue

Re: Hive webUrl issue

Re: Hive webUrl issue

Re: Permission denied as I am unable to delete a d...

Re: How many users connected to HiveServer2

Re: Tez UI WebService failed to start

Re: Connect Cloudera HIVE ODBC 2.6.4 to HiveServer...

Re: show tables not showing in beeline