I tried increasing the parallelism in LLAP by increasing the number of mappers by changing the tez.grouping.min-size. While I run the query I see the execution time is varying from 14sec to 40secs. I have set the orc block size to 1Mb and tez.grouping.min-size=2Mb. When the query runs for 40sec,the tasks which are making it delayed can all be seen on a single node. Is there a tradeoff in increasing the parallelism in LLAP?
Thank you for all your questions regarding LLAP and parallel processing. In general, tuning parallelized systems and code that runs on parallel systems can take some time to tune. A lot of this depends on the application and how much parallelism can be leveraged in a system and your given query and dataset. Please provide more information about your environment, as mentioned in your other questions. There are definitely some tradeoffs to increasing parallelism, and this depends on the size of your cluster, resources on each node, other apps running, JVM behvior, etc.