I am trying to enable LLAP, I am getting very poor memory allocations.
Memory per Daemon - 53248
In-Memory Cache per Daemon - 0
Number of executors per LLAP Daemon - 13
I even created a new queue but still no effect. Would appreciate any help
Can you please elaborate a bit more, as to what is the memory usage post enabling LLAP (used/total memory) ? also which HDP version you are using and cluster size ?
There are multiple params one should configure, like numbers bodes used by llap, number of llap daemons, llap_heap_size, memory cash per daemon, number of threads and some more like max memory for yarn container and tez container size.
The value of all will depend on the cluster config: like memory/node, cpu cores/node and number of nodes. So, please check all these params and let me know cluster details if you want some recommendations from my side.
Also, you can refer to this link which probably will clarify things for you: https://community.hortonworks.com/articles/149486/llap-sizing-and-setup.html
Thanks @nkumar for your response.
I have reverted my changes because there was no point in turning it ON as In-Memory Cache per Daemon was 0.
I have a 5 node heterogeneous cluster hence I allocated 2 nodes (Name Nodes) to be used by LLAP,
I do not know how to configure number of llap daemons.
I have HiveServer2 Heap Size =16GB and can increase it to 250GB. Metastore Heap Size = 2048 and again the same can be increased to 250GB and Client Heap Size=4096 and again the same can be increased to 250GB.
max memory for yarn container = 250GB, currently allocated 190GB for Node 1, 100GB for Node 2 and 60GB for the others.
I created an additional queue following the llap sizing and setup document but it did not help. I even set up seperate node 1 and 2 configs in Hive and Yarn. But it is not transfering the additional horsepower even I assign the new queue to the node 1 configuration(which has 250GB), it is still showing Memory per Daemon - 53248 and In-Memory Cache per Daemon - 0
Any help would be appreciated