Cloudera Community
manjilhk
user rank
Explorer
My Accepted Solutions
Show All

Re: Python3 connection to Kerberos Hbase thrift HT...

by Master Collaborator Master Collaborator in Support Questions
‎07-31-2021 10:12 PM
‎07-31-2021 10:12 PM
@manjilhk Thanks for sharing this awesome solution using THttpClient transport, can you let us know which CDH version are you at? In CDH6.x the TSaslClientTransport is working, but in CDP starter version there's some code changed to cause this transport failed to communicate with secured cluster. We have released hotfix to this issue, if below KB matches your issue please raise a Cloudera case to apply for this hotfix, or you need to wait for the future release 7.2.11 which will include this fix. Please see this KB that I posted: https://my.cloudera.com/knowledge/Cannot-connect-to-HBase-Thrift-from-Python-scripts-after?id=318921   - Will Xiao, Support Engineer Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, click on the thumbs up button. ... View more

Re: Hive CLI and Beeline jdbc:hive2 behave differe...

by Explorer in Support Questions
‎08-21-2018 10:08 AM
‎08-21-2018 10:08 AM
Update: It is found that .hiverc was used in hive user for HIVE CLI so the difference was found. hive.exec.scratchdir=/user/hive/scratch hive.exec.stagingdir=/user/hive/staging The issue is hdfs /user/hive directory is encrypted with Ranger and hdfs /tmp/hive directory is non-encrypted and can read/write by all user in hadoop group. hive-site.xml <property> <name>hive.security.authorization.sqlstd.confwhitelist.append</name> <value>hive\.exec\.scratchdir|hive\.exec\.stagingdir</value> <description>append conf property in white list followed by pipeline</description> </property> Restart the metastore and hiveserver. I tested with beeline with session level change . The execution is fast like HIVE CLI . hive.exec.scratchdir=/user/hive/scratch hive.exec.stagingdir=/user/hive/staging I tested with HIVE CLI with session level change. The execution is slow with MAP reduce Job for moving data. hive.exec.scratchdir=/tmp/hive/scratch hive.exec.stagingdir=/tmp/hive/staging So the root cause is data are encrypted in /user/hive and not encrypted in /tmp/hive. Solution is to make ssession level change to use same encryption zone. So below INFO log will be printed if the encryption zones are different. metadata.Hive: Copying source hdfs://edhcluster/tmp/hive/staging_hive_2018-08-07_16- 29-12_750_8973639287951385407-1/-ext-10000/000001_0 to hdfs://edhcluster/user/hive/warehouse/temp_tro/000001_0 because HDFS encryption zones are different. Thanks, Manjil ... View more
Contact Me
Online
Offline
Last Visited
‎07-05-2021 05:15 AM
Community Statistics
Member Since ‎07-19-2018 02:10 AM
Last Visited ‎07-05-2021 05:15 AM
Posts 8