I am running a very simple yarn application where the launch command is just a pipline of simple commands:
"launch_command": "echo XXXXX ; java -version ; hostname ; date ; ls -l /home ; sleep 6000",
When I look for the output in the logs it appears in the prelaunch log files. The output of the "java -version" commands appears in prelaunch.err and the output of the hostname, date, and ls commands appears in prelaunch.out.
I can see the launch command in the logs:
echo "Launching container"
exec /bin/bash -c "echo XXXXX ; java -version ; hostname ; date ; ls -l /home ; sleep 6000 1>/hadoop/yarn/log/application_1562080711961_0050/container_e12_1562080711961_0050_01_000002/stdout.txt 2>/hadoop/yarn/log/application_1562080711961_0050/container_e12_1562080711961_0050_01_000002/stderr.txt "
So why does the output go into prelaunch.err and prelaunch.out instead of stderr.txt and stdout.txt?
My application JSON file is:
{
"name": "java-version-app",
"version": "1.0.0",
"lifetime": "60",
"components" :
[
{
"name": "component0",
"launch_command": "echo XXXXX ; java -version ; hostname ; date ; ls -l /home ; sleep 600",
"number_of_containers": 1,
"resource":
{
"cpus": 1,
"memory": "256"
}
}
]
}
My HDP version is:
Hadoop 3.1.1.3.1.0.0-78
Source code repository git@github.com:hortonworks/hadoop.git -r e4f82af51faec922b4804d0232a637422ec29e64
Compiled by jenkins on 2018-12-06T13:34Z
Compiled with protoc 2.5.0
From source with checksum eab9fa2a6aa38c6362c66d8df75774
This command was run using /usr/hdp/3.1.0.0-78/hadoop/hadoop-common-3.1.1.3.1.0.0-78.jar
