Hive OutOfMemoryError: unable to create new native thread

[#1]

Seeing below exception on running Hive TPCDS data gen (https://github.com/hortonworks/hive-testbench) for a scale of ~500G.

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.lang.OutOfMemoryError: unable to create new native thread

Attached log for complete stacktrace.

Cluster Configuration :

16 Nodes / 12 Nodemanagers / 12 Datanodes

Per Node Config :

Cores : 40

Memory : 392GB

Ambari Configs changed from initial configs to improve performance :

Decided to set 10G as container size to utilise maximum cores per node (320G/10G = 32 containers using 1 Core/node. Hence ~32 Cores/node utilised)

YARN

yarn.nodemanager.resource.memory-mb = 329216 MB

yarn.scheduler.minimum-allocation-mb = 10240 MB

yarn.scheduler.maximum-allocation-mb = 329216 MB

MapReduce (All Heap Sizes : -Xmx8192m : 80% of container)

mapreduce.map.memory.mb = 10240 MB

mapreduce.reduce.memory.mb = 10240 MB

mapreduce.task.io.sort.mb = 1792 MB

yarn.app.mapreduce.am.resource.mb = 10240 MB

Hive

hive.tez.container.size = 10240MB

hive.auto.convert.join.noconditionaltask.size = 2027316838 B

hive.exec.reducers.bytes.per.reducer = 1073217536 B

Tez

tez.am.resource.memory.mb = 10240 MB

tez.am.resource.java.opts = -server -Xmx8192m

tez.task.resource.memory.mb = 10240 MB

tez.runtime.io.sort.mb = 2047 MB (~20% of container)

tez.runtime.unordered.output.buffer.size-mb = 768 MB (~10% of container)

tez.grouping.max-size = 2073741824 B

tez.grouping.min-size = 167772160 B

Any help would be greatly appreciated.
Referred https://community.cloudera.com/t5/Community-Articles/Demystify-Apache-Tez-Memory-Tuning-Step-by-Step... for some tuning values.

[#2]

Hi @ssulav ,

I see the below error in the HS2 logs shared:

ERROR [HiveServer2-Background-Pool: Thread-886]: SessionState (:()) - Vertex failed, vertexName=Map 1, vertexId=vertex_1571760131080_0019_1_00, diagnostics=[Task failed, taskId=task_1571760131080_0019_1_00_000380, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : java.lang.OutOfMemoryError: unable to create new native thread

The above error is reported by the Yarn application.

This error is not related to any Hadoop or Yarn or Hive configuration. Rather this is an error received by the OS for now able to create new threads for the process.

You need to check the ulimits of the Yarn user on the NodeManager and ResourceManager nodes.

Though, it is more likely this error is coming from the NodeManager running the above mentioned Task ID.

You can identify the host where the above Task is running by searching for the vertex or Task ID in the Yarn application logs.

The job would be running as Yarn user and check for similar errors in the nodemanager logs for the same host.

You can try increase the ulimit for "ulimit -u" option.

[#3]

Thanks for reply @rohitmalhotra 
User Limit for yarn is set to 65536.
Is there any recommended highest value or shall I just make it unlimited? (It can have consequences?)

Edit : I tried setting unlimited. Still seeing same error.

[#4]

You can try changing the limits from Ambari as well.

Under Ambari > Yarn Configs > Advanced:

Restart Yarn after increasing the limit.