Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎12-31-2014

CDH 5.3.0 container cannot be fetched because of NumberFormatException

Hi I am running spark inside YARN job. The spark job is running fine in CDH-5.0.0 but after the upgrdrae the it cannot fetch containers with the below errors. Looks like the container id is incorect and a string is present in a pace where it's expecting a number.

 

java.lang.IllegalArgumentException: Invalid ContainerId: container_e01_1420481081140_0006_01_000001

Caused by: java.lang.NumberFormatException: For input string: "e01"

 

Is this a bug?? Did you face something similar?

 

15/01/08 09:50:28 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT]
15/01/08 09:50:29 ERROR yarn.ApplicationMaster: Uncaught exception:
java.lang.IllegalArgumentException: Invalid ContainerId: container_e01_1420481081140_0006_01_000001
at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182)
at org.apache.spark.deploy.yarn.YarnRMClientImpl.getAttemptId(YarnRMClientImpl.scala:79)
at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:79)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$main$1.apply$mcV$sp(ApplicationMaster.scala:515)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:60)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:59)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:513)
at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
Caused by: java.lang.NumberFormatException: For input string: "e01"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:441)
at java.lang.Long.parseLong(Long.java:483)
at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137)
at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177)
... 11 more
15/01/08 09:50:29 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 10, (reason: Uncaught exception: Invalid ContainerId: container_e01_1420481081140_0006_01_000001)

New Contributor
Posts: 3
Registered: ‎12-31-2014

Re: CDH 5.3.0 container cannot be fetched because of NumberFormatException

Hey Guys, I donot have a spark assembly jar for CDH 5.3.0.

 

Can you help me to find out how to submit CDH-5.3.0 spark application in YARN?

Cloudera Employee
Posts: 366
Registered: ‎07-29-2013

Re: CDH 5.3.0 container cannot be fetched because of NumberFormatException

Have a look in the Cloudera repo:

 

https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/spark/

 

But, you do not need to bother finding or copying or deploying the assembly. Of course, it is already set up for you on the cluster. As assembly jar is nothing to do with your problem.

Posts: 1,903
Kudos: 435
Solutions: 307
Registered: ‎07-31-2013

Re: CDH 5.3.0 container cannot be fetched because of NumberFormatException

This issue can happen if you use a client hadoop library version older than 5.3.0 (such as 5.2.x, etc.) on a cluster server running 5.3.0 or higher. Ensuring your Spark libraries use the same CDH Hadoop (YARN) version as the server will resolve this.