Error: Error while processing statement: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.tez.TezTask. unable to create new native thread (state=08S01,code=-101)

[#1]

When i run a beeline query . It throws following excetion and some time query produces correct output

Found these on hiveserver2 logs

ERROR [HiveServer2-Background-Pool: Thread-160573]: operation.Operation (SQLOperation.java:run(209)) - Error running hive query: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.tez.TezTask. unable to create new native thread at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:156) at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71) at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method)

[#2]

check resources on node where this task failed, seems a resource problem there .. from log - unable to create new native thread at

[#3]

@chennuri gouri shankar

Can you please check if there are more errors in the logs, usually this would be caused by OutOfMemory

[#4]

This is what i can see in my logs

[#5 · Accepted solution]

@chennuri gouri shankar

Can you please check /etc/security/limits.conf and the check the configuration for the user running beeline?

Try to increase the nofile & nproc and see if it helps.

Thanks and Regards,

Sindhu

[#6]

Thanks sindhu It worked for me