Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

YARN containers failing with "/bin/bash: Argument list too long" exception.

YARN containers failing with "/bin/bash: Argument list too long" exception.

New Contributor

Hello,

I have an application that runs on top of YARN, when we use comparitely less number of partitions, it runs fine. However when we increase it to the upper limit, for us its 400, the containers start to fail.

With following exception :

Container id: container_1450377637401_0019_01_000107
Exit code: 1
Exception message: /hdfs5/yarn/nm/usercache/user1/appcache/application_1450377637401_0019/container_1450377637401_0019_01_000107/launch_container.sh: line 58: /bin/bash: Argument list too long
/hdfs5/yarn/nm/usercache/user1/appcache/application_1450377637401_0019/container_1450377637401_0019_01_000107/launch_container.sh: line 58: /bin/bash: Success

 

2 REPLIES 2

Re: YARN containers failing with "/bin/bash: Argument list too long" exception.

New Contributor
We've tried increasing ARG_MAX value at system level to 2621440, but no effect and it still fails.
Highlighted

Re: YARN containers failing with "/bin/bash: Argument list too long" exception.

New Contributor

set hive.script.operator.truncate.env to true