Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
Go to solution
Increase max row size in HIVE
Labels:
- Labels:
-
Apache Hive
Contributor
Created on ‎07-08-2021 01:50 AM - edited ‎07-08-2021 01:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a pyspark job with these configs:
self.spark = SparkSession.builder.appName("example") \
.config("hive.exec.dynamic.partition", "true") \
.config("hive.exec.dynamic.partition.mode", "nonstrict") \
.config("hive.exec.max.dynamic.partitions", "5000000") \
.config("hive.exec.max.dynamic.partitions.pernode", "1000000") \
.enableHiveSupport() \
.getOrCreate()
I can not find anywhere how to set a configuration to increase the max row size to 150mb. I found the command only in impala.
Can you please help me ?
Thanks in advance.
1 ACCEPTED SOLUTION
Expert Contributor
Created ‎07-11-2021 06:37 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1 REPLY 1
Expert Contributor
Created ‎07-11-2021 06:37 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
The max_row_size parameter is Impala specific parameter
