Support Questions
Find answers, ask questions, and share your expertise

Couldn't deploy Yarn session cluster | Flink in CDH 6.3.2

Couldn't deploy Yarn session cluster | Flink in CDH 6.3.2

New Contributor

hello I am trying to deploy a client flink app that reads from a kafka topic and print messages

 

1- error message

The program finished with the following exception:

org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:385)
at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:262)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:216)
at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1021)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1096)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1096)
Caused by: org.apache.flink.yarn.AbstractYarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.
Diagnostics from YARN: Application application_1609419738622_0006 failed 2 times in previous 10000 milliseconds due to AM Container for appattempt_1609419738622_0006_000002 exited with exitCode: 2
Failing this attempt.Diagnostics: [2021-01-10 11:14:00.079]Exception from container-launch.
Container id: container_1609419738622_0006_02_000001
Exit code: 2

[2021-01-10 11:14:00.108]Container exited with a non-zero exit code 2. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :

[2021-01-10 11:14:00.109]Container exited with a non-zero exit code 2. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :

For more detailed output, check the application tracking page: http://TSJ-DTM-CV-BD03:8088/cluster/app/application_1609419738622_0006 Then click on links to logs of each attempt.
. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1609419738622_0006
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.startAppMaster(AbstractYarnClusterDescriptor.java:1045)
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:507)
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:378)
... 9 more
21/01/10 11:14:00 INFO yarn.AbstractYarnClusterDescriptor: Cancelling deployment from Deployment Failure Hook
21/01/10 11:14:00 INFO client.RMProxy: Connecting to ResourceManager at TSJ-DTM-CV-BD03/xx.xx.xx.50:8032
21/01/10 11:14:00 INFO yarn.AbstractYarnClusterDescriptor: Killing YARN application
21/01/10 11:14:00 INFO impl.YarnClientImpl: Killed application application_1609419738622_0006
21/01/10 11:14:00 INFO yarn.AbstractYarnClusterDescriptor: Deleting files in hdfs://xxxxx/user/centos/.flink/application_1609419738622_0006.

 

2- command used to run the app

  flink run -m yarn-cluster /xxx_flink_PoC/flink_PoC/target/flink_PoC-0.1.jar

 

 

thank you