17/11/08 14:47:09 INFO SecurityManager: Changing view acls to: root
17/11/08 14:47:09 INFO SecurityManager: Changing modify acls to: root
17/11/08 14:47:09 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
17/11/08 14:47:09 INFO Client: Submitting application 4 to ResourceManager
17/11/08 14:47:09 INFO YarnClientImpl: Submitted application application_1510129660245_0004
17/11/08 14:47:09 INFO SchedulerExtensionServices: Starting Yarn extension services with app application_1510129660245_0004 and attemptId None
17/11/08 14:47:10 INFO Client: Application report for application_1510129660245_0004 (state: ACCEPTED)
17/11/08 14:47:10 INFO Client:
         client token: N/A
         diagnostics: AM container is launched, waiting for AM container to Register with RM
         ApplicationMaster host: N/A
         ApplicationMaster RPC port: -1
         queue: default
         start time: 1510132629142
         final status: UNDEFINED
         tracking URL: http://clustername:8088/proxy/application_1510129660245_0004/
         user: root
17/11/08 14:47:11 INFO Client: Application report for application_1510129660245_0004 (state: ACCEPTED)
17/11/08 14:47:12 INFO Client: Application report for application_1510129660245_0004 (state: ACCEPTED)
17/11/08 14:47:13 INFO Client: Application report for application_1510129660245_0004 (state: ACCEPTED)
17/11/08 14:47:14 INFO Client: Application report for application_1510129660245_0004 (state: FAILED)
17/11/08 14:47:14 INFO Client:
         client token: N/A
         diagnostics: Application application_1510129660245_0004 failed 2 times due to AM Container for appattempt_1510129660245_0004_000002 exited with  exitCode: 1
For more detailed output, check the application tracking page: http://clustername:8088/cluster/app/application_1510129660245_0004 Then click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e08_1510129660245_0004_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:944)
        at org.apache.hadoop.util.Shell.run(Shell.java:848)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1142)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)


Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
         ApplicationMaster host: N/A
         ApplicationMaster RPC port: -1
         queue: default
         start time: 1510132629142
         final status: FAILED
         tracking URL: http://clustername:8088/cluster/app/application_1510129660245_0004
         user: root
17/11/08 14:47:14 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:122)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:62)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:530)
        at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
        at com.samsung.cognitiv.dma.analyzer.DMACoordinatorJob.main(DMACoordinatorJob.java:92)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:750)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
17/11/08 14:47:14 INFO SparkUI: Stopped Spark web UI at http://107.110.7.59:4042
17/11/08 14:47:14 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
17/11/08 14:47:14 INFO YarnClientSchedulerBackend: Shutting down all executors
17/11/08 14:47:14 INFO YarnClientSchedulerBackend: Asking each executor to shut down
17/11/08 14:47:14 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
 services=List(),
 started=false)
17/11/08 14:47:14 INFO YarnClientSchedulerBackend: Stopped
17/11/08 14:47:14 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/11/08 14:47:14 INFO MemoryStore: MemoryStore cleared
17/11/08 14:47:14 INFO BlockManager: BlockManager stopped
17/11/08 14:47:14 INFO BlockManagerMaster: BlockManagerMaster stopped
17/11/08 14:47:14 WARN MetricsSystem: Stopping a MetricsSystem that is not running
17/11/08 14:47:14 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/11/08 14:47:14 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:122)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:62)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:530)
        at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
        at com.samsung.cognitiv.dma.analyzer.DMACoordinatorJob.main(DMACoordinatorJob.java:92)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:750)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
17/11/08 14:47:14 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
17/11/08 14:47:14 INFO ShutdownHookManager: Shutdown hook called
17/11/08 14:47:14 INFO ShutdownHookManager: Deleting directory /home/dma/tmp/spark-8ed1def9-519a-4fbd-a298-d482b3fbb9d0/httpd-f73e138f-8df5-4285-8a95-f63acb25f0f2
17/11/08 14:47:14 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
17/11/08 14:47:14 INFO ShutdownHookManager: Deleting directory /home/dma/tmp/spark-8ed1def9-519a-4fbd-a298-d482b3fbb9d0