Support Questions

Find answers, ask questions, and share your expertise

how to more hive performance!

Explorer

I have a hadoop cluster in google cloud but in recent days I have suffered with slow queries, using beeline and hive .. beeline internally and hive externally, I have below some screens that show me errors. I would like to know how to edit the memory allocated for java. Today is -Xmx512m -Xms512 ... and wanted to switch to -Xmx1024m -Xms1024 to validate if this can bring me more performance!

86543-photo-2018-08-10-09-54-35.jpg

86544-photo-2018-08-10-09-54-49.jpg

1 REPLY 1

@Hugo Cosme editing the JVM memory for a yarn container or app is likely done by editing your Yarn/MapReduce settings via Ambari to higher than 512mb. Similarly various HiveServer2 memory settings can be edited as well.

If you are trying to get Hive Performance (original post subject) I would recommend Hive LLAP if you are not already using it. You can search the HCC for many posts about LLAP configuration and setup.

If it is helpful here are some of my Settings For Hive, Yarn, Map Reduce:

HiveServer2: 16g
Metastore Heap Size: 8gb
Client heap Size: 4gb
Map/Join per Map Memory Threshold: 4 gb
Data Per Reducer: 4gb
Tez container Size: 12gb

Yarn Memory: 52gb
Container: 2560mb
Max Container: 45056MB

Map Memory: 12 gb
Reduce Memory: 22gb

If this answer is helpful, please choose Accept to mark the question answered.