Cloudera Community
sree_nv
user rank
user rank
Contributor

Re: Merge small files in pyspark for Hive table

by Contributor Contributor in Support Questions
‎01-07-2022 05:45 AM
‎01-07-2022 05:45 AM
@drgenious  1)If those small files are delta files, you can run compaction manually to merge all of them more details on manual compaction described in the link below: https://docs.cloudera.com/runtime/7.2.10/managing-hive/topics/hive_initiate_hive_compaction.html   2)Current Hive versions with RDBMS metastore backend should be able to handle 10 000+ partitions.   Thanks, Sree ... View more

Re: Disable spark2 thrift server

by Contributor Contributor in Support Questions
‎01-07-2022 05:40 AM
‎01-07-2022 05:40 AM
@Lo  Good Day!! Even though you submit spark-submit via JDBC the request goes through the HMS. HMS detects the type of client for interacting with it, for example, Hive or Spark, and acts accordingly. If you see any timeout requests in the logs you can adjust the below property. hive.metastore.client.socket.timeout=<to a higher value>  Check and let us know If this helped. ... View more

Re: Hi , i am facing issue when i am adding new se...

by Contributor Contributor in Support Questions
‎01-07-2022 05:31 AM
‎01-07-2022 05:31 AM
@NSandeep  Tez is a client which is mandatory to add before installing Hive on Tez.   Try to add Tez and then Hive on Tez it will be successful.   Check and let us know If this helped. https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/hive-introduction/topics/hive_installing_on_tez.html   Thanks, Sree ... View more
Contact Me
Online
Offline
Last Visited
‎05-10-2022 06:22 AM
Community Statistics
Member Since ‎02-10-2020 09:46 PM
Last Visited ‎05-10-2022 06:22 AM
Posts 64