Use the following tool to generate no of executors: https://rangareddy.github.io/SparkConfigurationGenerator/   In order to calculate the driver memory/executor memory we need to start with 1g, 2g, 4g, 8g .... and executor-cores you can set 3-5 and number of executor it will depend on data how much you are processing.   ... View more

Hi @pankshiv1809    To run application faster, we need to tune the resources, spark code and cluster. To solve any kind of performance issues, you need to go through the Spark UI and understand jobs, stages and executors. After that you need to tune the resources like driver and executor memory and no of executors and separate queue to process the data.    If you want to know other techniques raise the Cloudera case, we will help to further.  ... View more

GM Asish... Thanks for sharing difference between two parameter. Will take note on  tez.task.resource.memory.mb should be set < hive.tez.container.size    Appreciate for your support.   Regards, Pankaj shivankar ... View more