My destination is to test hive2 LLAP on my ambari 2.2.2.0 cluster. The service stack is: tez: 0.7.0.2.4 yarn: 2.7.1.2.4 hdfs: 2.7.1.2.4 hive: 1.2.1.2.4 ... I download apache-hive-2.3.2 and apache-tez-0.8.5 for test. After a few configuration steps for hive on tez, I met a unknown problem without any tip or Exception. Here is the application log: Container: container_e51_1511064572746_31935_01_000001 on datanode36.will.com_45454 ======================================================================================= LogType:directory.info Log Upload Time:Mon Dec 11 19:03:38 +0800 2017 LogLength:14351 Log Contents: ls -l: total 36 -rw-r--r-- 1 yarn hadoop 178 Dec 11 19:03 container_tokens -rwx------ 1 yarn hadoop 692 Dec 11 19:03 default_container_executor_session.sh -rwx------ 1 yarn hadoop 746 Dec 11 19:03 default_container_executor.sh lrwxrwxrwx 1 yarn hadoop 142 Dec 11 19:03 hive-exec-2.3.2-d0c675ed3fe95aae04b88516604f1b3eff1fe5bc7fb33073f97252dc7a384fbd.jar -> /server/hadoop/yarn/local/usercache/root/filecache/560002/hive-exec-2.3.2-d0c675ed3fe95aae04b88516604f1b3eff1fe5bc7fb33073f97252dc7a384fbd.jar -rwx------ 1 yarn hadoop 4878 Dec 11 19:03 launch_container.sh lrwxrwxrwx 1 yarn hadoop 106 Dec 11 19:03 tez-conf.pb -> /server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/filecache/11/tez-conf.pb lrwxrwxrwx 1 yarn hadoop 49 Dec 11 19:03 tezlib -> /server/hadoop/yarn/local/filecache/23/tez.tar.gz lrwxrwxrwx 1 yarn hadoop 125 Dec 11 19:03 tez.session.local-resources.pb -> /server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/filecache/10/tez.session.local-resources.pb drwx--x--- 2 yarn hadoop 4096 Dec 11 19:03 tmp find -L . -maxdepth 5 -ls: 2230569 4 drwx--x--- 3 yarn hadoop 4096 Dec 11 19:03 . 2230602 4 -rwx------ 1 yarn hadoop 746 Dec 11 19:03 ./default_container_executor.sh 2230593 88 -r-x------ 1 yarn hadoop 90060 Dec 11 19:03 ./tez-conf.pb 2230598 8 -rwx------ 1 yarn hadoop 4878 Dec 11 19:03 ./launch_container.sh 2230599 4 -rw-r--r-- 1 yarn hadoop 48 Dec 11 19:03 ./.launch_container.sh.crc 26607805 4 drwxr-xr-x 3 yarn hadoop 4096 Dec 11 15:45 ./tezlib 26607814 152 -r-xr-xr-x 1 yarn hadoop 154464 Apr 25 2016 ./tezlib/tez-tests-0.7.0.2.4.2.0-258.jar 26607816 28 -r-xr-xr-x 1 yarn hadoop 25622 Apr 25 2016 ./tezlib/tez-yarn-timeline-history-0.7.0.2.4.2.0-258.jar 26607811 272 -r-xr-xr-x 1 yarn hadoop 275362 Apr 25 2016 ./tezlib/tez-mapreduce-0.7.0.2.4.2.0-258.jar 26607810 532 -r-xr-xr-x 1 yarn hadoop 542434 Apr 25 2016 ./tezlib/tez-runtime-library-0.7.0.2.4.2.0-258.jar 26607813 1196 -r-xr-xr-x 1 yarn hadoop 1222496 Apr 25 2016 ./tezlib/tez-dag-0.7.0.2.4.2.0-258.jar 26607815 1600 -r-xr-xr-x 1 yarn hadoop 1636310 Apr 25 2016 ./tezlib/tez-ui-0.7.0.2.4.2.0-258.war 26607818 8 -r-xr-xr-x 1 yarn hadoop 7784 Apr 25 2016 ./tezlib/tez-yarn-timeline-history-with-acls-0.7.0.2.4.2.0-258.jar 26607806 868 -r-xr-xr-x 1 yarn hadoop 888761 Apr 25 2016 ./tezlib/tez-api-0.7.0.2.4.2.0-258.jar 26607819 8 -r-xr-xr-x 1 yarn hadoop 6334 Apr 25 2016 ./tezlib/tez-yarn-timeline-cache-plugin-0.7.0.2.4.2.0-258.jar 26607808 164 -r-xr-xr-x 1 yarn hadoop 164267 Apr 25 2016 ./tezlib/tez-runtime-internals-0.7.0.2.4.2.0-258.jar 26607807 68 -r-xr-xr-x 1 yarn hadoop 67195 Apr 25 2016 ./tezlib/tez-common-0.7.0.2.4.2.0-258.jar 26607820 24 -r-xr-xr-x 1 yarn hadoop 22125 Apr 25 2016 ./tezlib/tez-yarn-timeline-history-with-fs-0.7.0.2.4.2.0-258.jar 26607817 64 -r-xr-xr-x 1 yarn hadoop 62943 Apr 25 2016 ./tezlib/tez-history-parser-0.7.0.2.4.2.0-258.jar 26607812 48 -r-xr-xr-x 1 yarn hadoop 47672 Apr 25 2016 ./tezlib/tez-examples-0.7.0.2.4.2.0-258.jar 26607822 4 drwxr-xr-x 2 yarn hadoop 4096 Dec 11 15:45 ./tezlib/lib 26607866 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 23 2016 ./tezlib/lib/protobuf-java-2.5.0.jar 26607833 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 23 2016 ./tezlib/lib/stax-api-1.0.1.jar 26607871 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 23 2016 ./tezlib/lib/apacheds-kerberos-codec-2.0.0-M15.jar 26607852 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 23 2016 ./tezlib/lib/jackson-jaxrs-1.9.13.jar 26607858 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 23 2016 ./tezlib/lib/commons-digester-1.8.jar 26607929 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 23 2016 ./tezlib/lib/objenesis-2.1.jar 26607868 72 -r-xr-xr-x 1 yarn hadoop 71570 Apr 25 2016 ./tezlib/lib/hadoop-auth-2.7.1.2.4.2.0-258.jar 26607912 152 -r-xr-xr-x 1 yarn hadoop 153798 Apr 25 2016 ./tezlib/lib/hadoop-azure-2.7.1.2.4.2.0-258.jar 26607917 404 -r-xr-xr-x 1 yarn hadoop 412739 Apr 23 2016 ./tezlib/lib/commons-lang3-3.3.2.jar 26607836 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 23 2016 ./tezlib/lib/commons-cli-1.2.jar 26607841 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 23 2016 ./tezlib/lib/commons-collections-3.2.2.jar 26607842 148 -r-xr-xr-x 1 yarn hadoop 147952 Apr 23 2016 ./tezlib/lib/jersey-json-1.9.jar 26607831 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 23 2016 ./tezlib/lib/commons-lang-2.6.jar 26607865 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 23 2016 ./tezlib/lib/xz-1.0.jar 26607825 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 23 2016 ./tezlib/lib/log4j-1.2.17.jar 26607909 36 -r-xr-xr-x 1 yarn hadoop 33483 Apr 23 2016 ./tezlib/lib/jackson-annotations-2.2.3.jar 26607928 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 23 2016 ./tezlib/lib/javassist-3.18.1-GA.jar 26607859 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 23 2016 ./tezlib/lib/commons-beanutils-1.7.0.jar 26607830 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 23 2016 ./tezlib/lib/commons-io-2.4.jar 26607880 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 23 2016 ./tezlib/lib/htrace-core-3.1.0-incubating.jar 26607877 184 -r-xr-xr-x 1 yarn hadoop 186273 Apr 23 2016 ./tezlib/lib/curator-framework-2.7.1.jar 26607894 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 23 2016 ./tezlib/lib/aopalliance-1.0.jar 26607840 268 -r-xr-xr-x 1 yarn hadoop 273370 Apr 23 2016 ./tezlib/lib/commons-net-3.1.jar 26607895 16 -r-xr-xr-x 1 yarn hadoop 14786 Apr 23 2016 ./tezlib/lib/jersey-guice-1.9.jar 26607826 1612 -r-xr-xr-x 1 yarn hadoop 1648200 Apr 23 2016 ./tezlib/lib/guava-11.0.2.jar 26607924 100 -r-xr-xr-x 1 yarn hadoop 98771 Apr 25 2016 ./tezlib/lib/hadoop-yarn-registry-2.7.1.2.4.2.0-258.jar 26607889 448 -r-xr-xr-x 1 yarn hadoop 458739 Apr 23 2016 ./tezlib/lib/jersey-core-1.9.jar 26607873 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 23 2016 ./tezlib/lib/api-asn1-api-1.0.0-M20.jar 26607879 268 -r-xr-xr-x 1 yarn hadoop 270342 Apr 23 2016 ./tezlib/lib/curator-recipes-2.7.1.jar 26607898 8228 -r-xr-xr-x 1 yarn hadoop 8421957 Apr 25 2016 ./tezlib/lib/hadoop-hdfs-2.7.1.2.4.2.0-258.jar 26607905 532 -r-xr-xr-x 1 yarn hadoop 540751 Apr 23 2016 ./tezlib/lib/jetty-6.1.26.hwx.jar 26607872 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 23 2016 ./tezlib/lib/apacheds-i18n-2.0.0-M15.jar 26607911 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 23 2016 ./tezlib/lib/joda-time-2.9.3.jar 26607849 64 -r-xr-xr-x 1 yarn hadoop 62983 Apr 23 2016 ./tezlib/lib/activation-1.1.jar 26607864 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 23 2016 ./tezlib/lib/commons-compress-1.4.1.jar 26607835 20 -r-xr-xr-x 1 yarn hadoop 17371 Apr 25 2016 ./tezlib/lib/hadoop-annotations-2.7.1.2.4.2.0-258.jar 26607891 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 23 2016 ./tezlib/lib/guice-servlet-3.0.jar 26607846 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 23 2016 ./tezlib/lib/stax-api-1.0-2.jar 26607892 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 23 2016 ./tezlib/lib/guice-3.0.jar 26607927 336 -r-xr-xr-x 1 yarn hadoop 340884 Apr 23 2016 ./tezlib/lib/fst-2.24.jar 26607920 36 -r-xr-xr-x 1 yarn hadoop 34835 Apr 25 2016 ./tezlib/lib/hadoop-yarn-server-web-proxy-2.7.1.2.4.2.0-258.jar 26607851 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 23 2016 ./tezlib/lib/jackson-mapper-asl-1.9.13.jar 26607901 1740 -r-xr-xr-x 1 yarn hadoop 1779991 Apr 23 2016 ./tezlib/lib/netty-all-4.0.23.Final.jar 26607896 184 -r-xr-xr-x 1 yarn hadoop 185401 Apr 25 2016 ./tezlib/lib/hadoop-yarn-client-2.7.1.2.4.2.0-258.jar 26607903 192 -r-xr-xr-x 1 yarn hadoop 194354 Apr 23 2016 ./tezlib/lib/xml-apis-1.3.04.jar 26607922 408 -r-xr-xr-x 1 yarn hadoop 413811 Apr 25 2016 ./tezlib/lib/hadoop-yarn-server-common-2.7.1.2.4.2.0-258.jar 26607890 128 -r-xr-xr-x 1 yarn hadoop 130458 Apr 23 2016 ./tezlib/lib/jersey-client-1.9.jar 26607899 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 23 2016 ./tezlib/lib/okhttp-2.4.0.jar 26607876 1172 -r-xr-xr-x 1 yarn hadoop 1199572 Apr 23 2016 ./tezlib/lib/netty-3.6.2.Final.jar 26607884 1672 -r-xr-xr-x 1 yarn hadoop 1710975 Apr 25 2016 ./tezlib/lib/hadoop-yarn-common-2.7.1.2.4.2.0-258.jar 26607908 848 -r-xr-xr-x 1 yarn hadoop 865838 Apr 23 2016 ./tezlib/lib/jackson-databind-2.2.3.jar 26607823 12 -r-xr-xr-x 1 yarn hadoop 8869 Apr 23 2016 ./tezlib/lib/slf4j-log4j12-1.7.5.jar 26607906 104 -r-xr-xr-x 1 yarn hadoop 104578 Apr 25 2016 ./tezlib/lib/hadoop-aws-2.7.1.2.4.2.0-258.jar 26607824 28 -r-xr-xr-x 1 yarn hadoop 26084 Apr 23 2016 ./tezlib/lib/slf4j-api-1.7.5.jar 26607886 176 -r-xr-xr-x 1 yarn hadoop 177487 Apr 23 2016 ./tezlib/lib/jetty-util-6.1.26.hwx.jar 26607902 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 23 2016 ./tezlib/lib/xercesImpl-2.9.1.jar 26607870 224 -r-xr-xr-x 1 yarn hadoop 227275 Apr 23 2016 ./tezlib/lib/httpcore-4.2.4.jar 26607853 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 23 2016 ./tezlib/lib/jackson-xc-1.9.13.jar 26607860 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 23 2016 ./tezlib/lib/commons-beanutils-core-1.8.0.jar 26607881 2060 -r-xr-xr-x 1 yarn hadoop 2108699 Apr 25 2016 ./tezlib/lib/hadoop-yarn-api-2.7.1.2.4.2.0-258.jar 26607918 1500 -r-xr-xr-x 1 yarn hadoop 1533812 Apr 25 2016 ./tezlib/lib/hadoop-mapreduce-client-core-2.7.1.2.4.2.0-258.jar 26607910 11672 -r-xr-xr-x 1 yarn hadoop 11948376 Apr 23 2016 ./tezlib/lib/aws-java-sdk-1.7.4.jar 26607839 60 -r-xr-xr-x 1 yarn hadoop 58160 Apr 23 2016 ./tezlib/lib/commons-codec-1.4.jar 26607854 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 23 2016 ./tezlib/lib/commons-logging-1.1.3.jar 26607834 3420 -r-xr-xr-x 1 yarn hadoop 3498444 Apr 25 2016 ./tezlib/lib/hadoop-common-2.7.1.2.4.2.0-258.jar 26607904 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 23 2016 ./tezlib/lib/leveldbjni-all-1.8.jar 26607913 652 -r-xr-xr-x 1 yarn hadoop 666763 Apr 23 2016 ./tezlib/lib/azure-storage-2.2.0.jar 26607844 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 23 2016 ./tezlib/lib/jaxb-api-2.2.2.jar 26607878 68 -r-xr-xr-x 1 yarn hadoop 69500 Apr 23 2016 ./tezlib/lib/curator-client-2.7.1.jar 26607838 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 23 2016 ./tezlib/lib/xmlenc-0.52.jar 26607863 976 -r-xr-xr-x 1 yarn hadoop 995968 Apr 23 2016 ./tezlib/lib/snappy-java-1.0.4.1.jar 26607856 192 -r-xr-xr-x 1 yarn hadoop 192699 Apr 23 2016 ./tezlib/lib/jackson-core-2.2.3.jar 26607827 36 -r-xr-xr-x 1 yarn hadoop 33031 Apr 23 2016 ./tezlib/lib/jsr305-2.0.3.jar 26607869 424 -r-xr-xr-x 1 yarn hadoop 433368 Apr 23 2016 ./tezlib/lib/httpclient-4.2.5.jar 26607843 872 -r-xr-xr-x 1 yarn hadoop 890168 Apr 23 2016 ./tezlib/lib/jaxb-impl-2.2.3-1.jar 26607900 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 23 2016 ./tezlib/lib/okio-1.4.0.jar 26607926 244 -r-xr-xr-x 1 yarn hadoop 246782 Apr 25 2016 ./tezlib/lib/hadoop-yarn-server-applicationhistoryservice-2.7.1.2.4.2.0-258.jar 26607862 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 23 2016 ./tezlib/lib/paranamer-2.3.jar 26607855 572 -r-xr-xr-x 1 yarn hadoop 582965 Apr 23 2016 ./tezlib/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar 26607893 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 23 2016 ./tezlib/lib/javax.inject-1.jar 26607861 300 -r-xr-xr-x 1 yarn hadoop 303139 Apr 25 2016 ./tezlib/lib/avro-1.7.4.jar 26607850 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 23 2016 ./tezlib/lib/jackson-core-asl-1.9.13.jar 26607867 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 23 2016 ./tezlib/lib/gson-2.2.4.jar 26607875 776 -r-xr-xr-x 1 yarn hadoop 794058 Apr 25 2016 ./tezlib/lib/zookeeper-3.4.6.2.4.2.0-258.jar 26607874 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 23 2016 ./tezlib/lib/api-util-1.0.0-M20.jar 26607919 736 -r-xr-xr-x 1 yarn hadoop 751873 Apr 25 2016 ./tezlib/lib/hadoop-mapreduce-client-common-2.7.1.2.4.2.0-258.jar 26607925 132 -r-xr-xr-x 1 yarn hadoop 134133 Apr 23 2016 ./tezlib/lib/servlet-api-2.5-20081211.jar 26607897 736 -r-xr-xr-x 1 yarn hadoop 751238 Apr 23 2016 ./tezlib/lib/commons-collections4-4.1.jar 26607832 80 -r-xr-xr-x 1 yarn hadoop 81743 Apr 23 2016 ./tezlib/lib/jettison-1.3.4.jar 26607930 60 -r-xr-xr-x 1 yarn hadoop 60819 Apr 25 2016 ./tezlib/lib/hadoop-yarn-server-timeline-plugins-2.7.1.2.4.2.0-258.jar 26607857 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 23 2016 ./tezlib/lib/commons-configuration-1.6.jar 26607885 104 -r-xr-xr-x 1 yarn hadoop 105112 Apr 23 2016 ./tezlib/lib/servlet-api-2.5.jar 26607837 1564 -r-xr-xr-x 1 yarn hadoop 1599627 Apr 23 2016 ./tezlib/lib/commons-math3-3.1.1.jar 2230597 4 -rw-r--r-- 1 yarn hadoop 12 Dec 11 19:03 ./.container_tokens.crc 2230596 4 -rw-r--r-- 1 yarn hadoop 178 Dec 11 19:03 ./container_tokens 2230600 4 -rwx------ 1 yarn hadoop 692 Dec 11 19:03 ./default_container_executor_session.sh 2230603 4 -rw-r--r-- 1 yarn hadoop 16 Dec 11 19:03 ./.default_container_executor.sh.crc 2230585 33412 -r-x------ 1 yarn hadoop 34210517 Dec 11 19:03 ./hive-exec-2.3.2-d0c675ed3fe95aae04b88516604f1b3eff1fe5bc7fb33073f97252dc7a384fbd.jar 2230595 4 drwx--x--- 2 yarn hadoop 4096 Dec 11 19:03 ./tmp 2230590 4 -r-x------ 1 yarn hadoop 456 Dec 11 19:03 ./tez.session.local-resources.pb 2230601 4 -rw-r--r-- 1 yarn hadoop 16 Dec 11 19:03 ./.default_container_executor_session.sh.crc broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Mon Dec 11 19:03:38 +0800 2017 LogLength:4878 Log Contents: #!/bin/bash export LOCAL_DIRS="/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935" export APPLICATION_WEB_PROXY_BASE="/proxy/application_1511064572746_31935" export HADOOP_CONF_DIR="/usr/hdp/current/hadoop-client/conf" export MAX_APP_ATTEMPTS="2" export TEZ_CLIENT_VERSION="0.8.5" export NM_HTTP_PORT="8042" export JAVA_HOME="/server/java/jdk1.8.0_60" export LOG_DIRS="/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="root" export HADOOP_YARN_HOME="/usr/hdp/current/hadoop-yarn-nodemanager" export CLASSPATH="/usr/hdp/2.4.2.0-258/hadoop/lib/hadoop-lzo-0.6.0.2.4.2.0-258.jar:/etc/hadoop/conf/secure:$PWD:$PWD/*:$PWD/tezlib/*:$PWD/tezlib/lib/*:$HADOOP_CONF_DIR:" export APP_SUBMIT_TIME_ENV="1512990207659" export NM_HOST="datanode36.will.com" export HADOOP_TOKEN_FILE_LOCATION="/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export LD_LIBRARY_PATH="$PWD:/usr/hdp/2.4.2.0-258/hadoop/lib/native:/usr/hdp/2.4.2.0-258/hadoop/lib/native/Linux-amd64-64:$PWD:/usr/hdp/2.4.2.0-258/hadoop/lib/native:/usr/hdp/2.4.2.0-258/hadoop/lib/native/Linux-amd64-64:$HADOOP_COMMON_HOME/lib/native/" export LOGNAME="root" export JVM_PID="$$" export PWD="/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001" export HOME="/home/" export CONTAINER_ID="container_e51_1511064572746_31935_01_000001" export MALLOC_ARENA_MAX="4" ln -sf "/server/hadoop/yarn/local/usercache/root/filecache/560002/hive-exec-2.3.2-d0c675ed3fe95aae04b88516604f1b3eff1fe5bc7fb33073f97252dc7a384fbd.jar" "hive-exec-2.3.2-d0c675ed3fe95aae04b88516604f1b3eff1fe5bc7fb33073f97252dc7a384fbd.jar" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/filecache/10/tez.session.local-resources.pb" "tez.session.local-resources.pb" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/filecache/11/tez-conf.pb" "tez-conf.pb" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/server/hadoop/yarn/local/filecache/23/tez.tar.gz" "tezlib" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/launch_container.sh" chmod 640 "/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/directory.info" ls -l 1>>"/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/directory.info" find -L . -maxdepth 5 -ls 1>>"/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/directory.info" exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx409m -Djava.io.tmpdir=$PWD/tmp -server -Djava.net.preferIPv4Stack=true -Dhdp.version=2.4.2.0-258 -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001 -Dtez.root.logger=INFO,CLA -Dsun.nio.ch.bugLevel='' org.apache.tez.dag.app.DAGAppMaster --session 1>/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/stdout 2>/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/stderr " hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh LogType:stderr Log Upload Time:Mon Dec 11 19:03:38 +0800 2017 LogLength:0 Log Contents: End of LogType:stderr LogType:stdout Log Upload Time:Mon Dec 11 19:03:38 +0800 2017 LogLength:6892 Log Contents: 0.614: [GC pause (G1 Evacuation Pause) (young), 0.0104031 secs] [Parallel Time: 5.4 ms, GC Workers: 8] [GC Worker Start (ms): Min: 614.0, Avg: 614.1, Max: 614.2, Diff: 0.2] [Ext Root Scanning (ms): Min: 0.5, Avg: 1.1, Max: 2.5, Diff: 2.1, Sum: 9.1] [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0] [Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0] [Scan RS (ms): Min: 0.0, Avg: 0.3, Max: 2.2, Diff: 2.2, Sum: 2.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.2, Max: 0.7, Diff: 0.7, Sum: 1.4] [Object Copy (ms): Min: 1.5, Avg: 2.9, Max: 3.8, Diff: 2.3, Sum: 22.8] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.2, Diff: 0.2, Sum: 1.2] [Termination Attempts: Min: 1, Avg: 25.9, Max: 44, Diff: 43, Sum: 207] [GC Worker Other (ms): Min: 0.0, Avg: 0.5, Max: 0.9, Diff: 0.8, Sum: 4.4] [GC Worker Total (ms): Min: 5.1, Avg: 5.2, Max: 5.2, Diff: 0.2, Sum: 41.3] [GC Worker End (ms): Min: 619.3, Avg: 619.3, Max: 619.3, Diff: 0.0] [Code Root Fixup: 0.1 ms] [Code Root Purge: 0.0 ms] [Clear CT: 0.3 ms] [Other: 4.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 3.7 ms] [Ref Enq: 0.0 ms] [Redirty Cards: 0.4 ms] [Humongous Register: 0.1 ms] [Humongous Reclaim: 0.0 ms] [Free CSet: 0.1 ms] [Eden: 24.0M(24.0M)->0.0B(39.0M) Survivors: 0.0B->3072.0K Heap: 24.0M(410.0M)->3743.5K(410.0M)] [Times: user=0.03 sys=0.00, real=0.01 secs] 1.318: [GC pause (G1 Evacuation Pause) (young), 0.0142482 secs] [Parallel Time: 10.2 ms, GC Workers: 8] [GC Worker Start (ms): Min: 1318.6, Avg: 1319.3, Max: 1320.3, Diff: 1.7] [Ext Root Scanning (ms): Min: 0.0, Avg: 0.9, Max: 3.2, Diff: 3.2, Sum: 7.4] [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.2, Diff: 0.2, Sum: 0.2] [Processed Buffers: Min: 0, Avg: 0.1, Max: 1, Diff: 1, Sum: 1] [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1] [Code Root Scanning (ms): Min: 0.0, Avg: 0.6, Max: 2.5, Diff: 2.5, Sum: 4.7] [Object Copy (ms): Min: 5.8, Avg: 7.5, Max: 8.4, Diff: 2.5, Sum: 60.2] [Termination (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.7] [Termination Attempts: Min: 1, Avg: 90.1, Max: 114, Diff: 113, Sum: 721] [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.2] [GC Worker Total (ms): Min: 8.2, Avg: 9.2, Max: 9.9, Diff: 1.7, Sum: 73.6] [GC Worker End (ms): Min: 1328.5, Avg: 1328.5, Max: 1328.5, Diff: 0.0] [Code Root Fixup: 0.5 ms] [Code Root Purge: 0.1 ms] [Clear CT: 0.3 ms] [Other: 3.2 ms] [Choose CSet: 0.0 ms] [Ref Proc: 2.4 ms] [Ref Enq: 0.0 ms] [Redirty Cards: 0.2 ms] [Humongous Register: 0.1 ms] [Humongous Reclaim: 0.0 ms] [Free CSet: 0.2 ms] [Eden: 39.0M(39.0M)->0.0B(240.0M) Survivors: 3072.0K->6144.0K Heap: 42.7M(410.0M)->8409.5K(410.0M)] [Times: user=0.07 sys=0.01, real=0.02 secs] 2.443: [GC pause (Metadata GC Threshold) (young) (initial-mark), 0.0219268 secs] [Parallel Time: 12.5 ms, GC Workers: 8] [GC Worker Start (ms): Min: 2443.8, Avg: 2443.9, Max: 2444.1, Diff: 0.3] [Ext Root Scanning (ms): Min: 1.0, Avg: 2.1, Max: 3.0, Diff: 2.0, Sum: 16.6] [Update RS (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 0.6] [Processed Buffers: Min: 0, Avg: 0.4, Max: 1, Diff: 1, Sum: 3] [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.0, Sum: 0.2] [Code Root Scanning (ms): Min: 0.0, Avg: 0.4, Max: 1.3, Diff: 1.3, Sum: 2.9] [Object Copy (ms): Min: 7.8, Avg: 9.5, Max: 10.9, Diff: 3.1, Sum: 75.7] [Termination (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.5] [Termination Attempts: Min: 1, Avg: 52.5, Max: 66, Diff: 65, Sum: 420] [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.2] [GC Worker Total (ms): Min: 12.0, Avg: 12.1, Max: 12.2, Diff: 0.3, Sum: 96.9] [GC Worker End (ms): Min: 2456.0, Avg: 2456.0, Max: 2456.1, Diff: 0.0] [Code Root Fixup: 0.6 ms] [Code Root Purge: 0.0 ms] [Clear CT: 0.4 ms] [Other: 8.4 ms] [Choose CSet: 0.0 ms] [Ref Proc: 7.4 ms] [Ref Enq: 0.1 ms] [Redirty Cards: 0.3 ms] [Humongous Register: 0.0 ms] [Humongous Reclaim: 0.0 ms] [Free CSet: 0.2 ms] [Eden: 80.0M(240.0M)->0.0B(123.0M) Survivors: 6144.0K->18.0M Heap: 88.2M(410.0M)->20.0M(410.0M)] [Times: user=0.09 sys=0.01, real=0.02 secs] 2.466: [GC concurrent-root-region-scan-start] 2.473: [GC concurrent-root-region-scan-end, 0.0071636 secs] 2.473: [GC concurrent-mark-start] 2.473: [GC concurrent-mark-end, 0.0003308 secs] 2.473: [GC remark 2.473: [Finalize Marking, 0.0010916 secs] 2.475: [GC ref-proc, 0.0001865 secs] 2.475: [Unloading, 0.0049696 secs], 0.0067141 secs] [Times: user=0.05 sys=0.01, real=0.00 secs] 2.481: [GC cleanup 21M->18M(410M), 0.0012542 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2.482: [GC concurrent-cleanup-start] 2.482: [GC concurrent-cleanup-end, 0.0000658 secs] 3.604: [GC pause (G1 Evacuation Pause) (young), 0.0290309 secs] [Parallel Time: 19.9 ms, GC Workers: 8] [GC Worker Start (ms): Min: 3604.0, Avg: 3604.2, Max: 3604.3, Diff: 0.2] [Ext Root Scanning (ms): Min: 1.4, Avg: 2.3, Max: 4.2, Diff: 2.8, Sum: 18.7] [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0] [Processed Buffers: Min: 0, Avg: 0.4, Max: 3, Diff: 3, Sum: 3] [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.4] [Code Root Scanning (ms): Min: 0.0, Avg: 4.1, Max: 11.3, Diff: 11.3, Sum: 32.4] [Object Copy (ms): Min: 6.0, Avg: 12.6, Max: 17.3, Diff: 11.3, Sum: 100.6] [Termination (ms): Min: 0.0, Avg: 0.6, Max: 0.7, Diff: 0.7, Sum: 4.5] [Termination Attempts: Min: 1, Avg: 85.9, Max: 160, Diff: 159, Sum: 687] [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.2] [GC Worker Total (ms): Min: 19.5, Avg: 19.6, Max: 19.7, Diff: 0.2, Sum: 156.8] [GC Worker End (ms): Min: 3623.7, Avg: 3623.8, Max: 3623.8, Diff: 0.0] [Code Root Fixup: 0.9 ms] [Code Root Purge: 0.0 ms] [Clear CT: 0.4 ms] [Other: 7.8 ms] [Choose CSet: 0.0 ms] [Ref Proc: 6.4 ms] [Ref Enq: 0.0 ms] [Redirty Cards: 0.3 ms] [Humongous Register: 0.0 ms] [Humongous Reclaim: 0.0 ms] [Free CSet: 0.7 ms] [Eden: 123.0M(123.0M)->0.0B(231.0M) Survivors: 18.0M->15.0M Heap: 140.8M(410.0M)->14.5M(410.0M)] [Times: user=0.18 sys=0.01, real=0.03 secs] Heap garbage-first heap total 419840K, used 105983K [0x00000000e6600000, 0x00000000e6700cd0, 0x0000000100000000) region size 1024K, 105 young (107520K), 15 survivors (15360K) Metaspace used 31350K, capacity 31598K, committed 31872K, reserved 1077248K class space used 3781K, capacity 3841K, committed 3968K, reserved 1048576K End of LogType:stdout LogType:syslog Log Upload Time:Mon Dec 11 19:03:38 +0800 2017 LogLength:17232 Log Contents: 2017-12-11 19:03:28,655 [INFO] [main] |app.DAGAppMaster|: Creating DAGAppMaster for applicationId=application_1511064572746_31935, attemptNum=1, AMContainerId=container_e51_1511064572746_31935_01_000001, jvmPid=32279, userFromEnv=root, cliSessionOption=true, pwd=/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001, localDirs=/server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935, logDirs=/server/hadoop/yarn/log/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001 2017-12-11 19:03:29,409 [INFO] [main] |app.DAGAppMaster|: Created DAGAppMaster for application appattempt_1511064572746_31935_000001, versionInfo=[ component=tez-dag, version=0.7.0.2.4.2.0-258, revision=fa554fdce4e3495e09a310e0a32bb34ccc5946ad, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160425-0638 ] 2017-12-11 19:03:29,428 [INFO] [main] |app.DAGAppMaster|: Comparing client version with AM version, clientVersion=0.8.5, AMVersion=0.7.0.2.4.2.0-258 2017-12-11 19:03:29,430 [ERROR] [main] |app.DAGAppMaster|: Incompatible versions found, clientVersion=0.8.5, AMVersion=0.7.0.2.4.2.0-258 2017-12-11 19:03:30,524 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap 2017-12-11 19:03:30,525 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker 2017-12-11 19:03:30,527 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler 2017-12-11 19:03:30,527 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher 2017-12-11 19:03:30,528 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher 2017-12-11 19:03:30,528 [INFO] [main] |app.DAGAppMaster|: Using concurrent dispatcher: false 2017-12-11 19:03:30,529 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher 2017-12-11 19:03:30,530 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher 2017-12-11 19:03:30,530 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler 2017-12-11 19:03:30,530 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler 2017-12-11 19:03:30,559 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerEventHandler 2017-12-11 19:03:30,561 [INFO] [main] |common.AsyncDispatcher|: Registering class org.apache.tez.dag.app.rm.NMCommunicatorEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherImpl 2017-12-11 19:03:30,564 [INFO] [main] |node.AMNodeTracker|: blacklistDisablePercent is 33, blacklistingEnabled: true, maxTaskFailuresPerNode: 10 2017-12-11 19:03:30,566 [INFO] [main] |web.WebUIService|: Tez UI History URL: http://schedule.will.com:8181/tez-ui-0.8.5?viewPath=%2F%23%2Ftez-app%2Fapplication_1511064572746_31935 2017-12-11 19:03:30,568 [INFO] [main] |launcher.ContainerLauncherImpl|: Upper limit on the thread pool size is 500 2017-12-11 19:03:30,568 [INFO] [main] |history.HistoryEventHandler|: Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService 2017-12-11 19:03:30,925 [INFO] [main] |impl.TimelineClientImpl|: Timeline service address: http://datanode04.will.com:8188/ws/v1/timeline/ 2017-12-11 19:03:30,926 [INFO] [main] |ats.ATSV15HistoryLoggingService|: Initializing ATSV15HistoryLoggingService with , maxPollingTime(ms)=10, waitTimeForShutdown(ms)=-1, TimelineACLManagerClass=org.apache.tez.dag.history.ats.acls.ATSV15HistoryACLPolicyManager 2017-12-11 19:03:31,040 [INFO] [main] |impl.TimelineClientImpl|: Timeline service address: http://datanode04.will.com:8188/ws/v1/timeline/ 2017-12-11 19:03:31,058 [INFO] [main] |recovery.RecoveryService|: RecoveryService initialized with recoveryPath=hdfs://dd/tmp/hive2/root/_tez_session_dir/656ffd05-b7b9-433c-9fa8-b8655dc9f840/.tez/application_1511064572746_31935/recovery/1, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100 2017-12-11 19:03:31,072 [INFO] [ServiceThread:org.apache.tez.dag.app.launcher.ContainerLauncherImpl] |impl.ContainerManagementProtocolProxy|: yarn.client.max-cached-nodemanagers-proxies : 0 2017-12-11 19:03:31,077 [INFO] [ServiceThread:org.apache.tez.dag.app.TaskAttemptListenerImpTezDag] |ipc.CallQueueManager|: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2017-12-11 19:03:31,077 [INFO] [ServiceThread:DAGClientRPCServer] |ipc.CallQueueManager|: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2017-12-11 19:03:31,092 [INFO] [Socket Reader #1 for port 38848] |ipc.Server|: Starting Socket Reader #1 for port 38848 2017-12-11 19:03:31,092 [INFO] [Socket Reader #1 for port 34543] |ipc.Server|: Starting Socket Reader #1 for port 34543 2017-12-11 19:03:31,119 [INFO] [IPC Server Responder] |ipc.Server|: IPC Server Responder: starting 2017-12-11 19:03:31,119 [INFO] [IPC Server listener on 34543] |ipc.Server|: IPC Server listener on 34543: starting 2017-12-11 19:03:31,120 [INFO] [IPC Server Responder] |ipc.Server|: IPC Server Responder: starting 2017-12-11 19:03:31,120 [INFO] [IPC Server listener on 38848] |ipc.Server|: IPC Server listener on 38848: starting 2017-12-11 19:03:31,122 [INFO] [ServiceThread:DAGClientRPCServer] |client.DAGClientServer|: Instantiated DAGClientRPCServer at datanode36.will.com/10.2.19.142:34543 2017-12-11 19:03:31,137 [INFO] [ServiceThread:org.apache.tez.dag.app.TaskAttemptListenerImpTezDag] |app.TaskAttemptListenerImpTezDag|: Instantiated TaskAttemptListener RPC at datanode36.will.com/10.2.19.142:38848 2017-12-11 19:03:31,184 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |mortbay.log|: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2017-12-11 19:03:31,196 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |server.AuthenticationFilter|: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2017-12-11 19:03:31,205 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpRequestLog|: Http request log for http.requests. is not defined 2017-12-11 19:03:31,213 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpServer2|: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2017-12-11 19:03:31,219 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpServer2|: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context 2017-12-11 19:03:31,219 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpServer2|: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static 2017-12-11 19:03:31,224 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpServer2|: adding path spec: /* 2017-12-11 19:03:31,608 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |webapp.WebApps|: Registered webapp guice modules 2017-12-11 19:03:31,610 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |http.HttpServer2|: Jetty bound to port 44877 2017-12-11 19:03:31,611 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |mortbay.log|: jetty-6.1.26.hwx 2017-12-11 19:03:31,652 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |mortbay.log|: Extract jar:file:/server/hadoop/yarn/local/filecache/23/tez.tar.gz/lib/hadoop-yarn-common-2.7.1.2.4.2.0-258.jar!/webapps/ to /server/hadoop/yarn/local/usercache/root/appcache/application_1511064572746_31935/container_e51_1511064572746_31935_01_000001/tmp/Jetty_0_0_0_0_44877_webapps____.3plv/webapp 2017-12-11 19:03:31,907 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |mortbay.log|: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:44877 2017-12-11 19:03:31,908 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |webapp.WebApps|: Web app started at 44877 2017-12-11 19:03:31,908 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |web.WebUIService|: Instantiated WebUIService at http://datanode36.will.com:44877/ui/ 2017-12-11 19:03:31,956 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerEventHandler] |rm.YarnTaskSchedulerService|: YarnTaskScheduler initialized with configuration: maxRMHeartbeatInterval: 250, containerReuseEnabled: true, reuseRackLocal: true, reuseNonLocal: false, localitySchedulingDelay: 250, preemptionPercentage: 10, preemptionMaxWaitTime: 60000, numHeartbeatsBetweenPreemptions: 3, idleContainerMinTimeout: 10000, idleContainerMaxTimeout: 20000, sessionMinHeldContainers: 0 2017-12-11 19:03:32,004 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerEventHandler] |client.RMProxy|: Connecting to ResourceManager at datanode02.will.com/10.2.19.83:8030 2017-12-11 19:03:32,084 [INFO] [main] |rm.TaskSchedulerEventHandler|: TaskScheduler notified that it should unregister from RM 2017-12-11 19:03:32,084 [INFO] [main] |app.DAGAppMaster|: DAGAppMasterShutdownHandler invoked 2017-12-11 19:03:32,084 [INFO] [main] |app.DAGAppMaster|: Handling DAGAppMaster shutdown 2017-12-11 19:03:32,086 [INFO] [AMShutdownThread] |app.DAGAppMaster|: Sleeping for 5 seconds before shutting down 2017-12-11 19:03:32,127 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:32,127 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: Allocated: <memory:0, vCores:0> Free: <memory:0, vCores:0> pendingRequests: 0 delayedContainers: 0 heartbeats: 1 lastPreemptionHeartbeat: 0 2017-12-11 19:03:32,129 [INFO] [Dispatcher thread {Central}] |node.AMNodeTracker|: Num cluster nodes = 25 2017-12-11 19:03:32,376 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:32,627 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:32,879 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:33,131 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:33,382 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:33,634 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:33,886 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:34,138 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:34,389 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:34,641 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:34,893 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:35,145 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:35,397 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:35,649 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:35,902 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:36,153 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:36,405 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:36,657 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:36,909 [INFO] [AMRM Callback Handler Thread] |rm.YarnTaskSchedulerService|: App total resource memory: 0 cpu: 0 taskAllocations: 0 2017-12-11 19:03:37,086 [INFO] [AMShutdownThread] |app.DAGAppMaster|: Calling stop for all the services 2017-12-11 19:03:37,086 [INFO] [AMShutdownThread] |rm.YarnTaskSchedulerService|: Initiating stop of YarnTaskScheduler 2017-12-11 19:03:37,086 [INFO] [AMShutdownThread] |rm.YarnTaskSchedulerService|: Releasing held containers 2017-12-11 19:03:37,087 [INFO] [AMShutdownThread] |rm.YarnTaskSchedulerService|: Removing all pending taskRequests 2017-12-11 19:03:37,088 [INFO] [AMShutdownThread] |history.HistoryEventHandler|: Stopping HistoryEventHandler 2017-12-11 19:03:37,088 [INFO] [AMShutdownThread] |recovery.RecoveryService|: Stopping RecoveryService 2017-12-11 19:03:37,088 [INFO] [AMShutdownThread] |recovery.RecoveryService|: Handle the remaining events in queue, queue size=0 2017-12-11 19:03:37,088 [INFO] [RecoveryEventHandlingThread] |recovery.RecoveryService|: EventQueue take interrupted. Returning 2017-12-11 19:03:37,089 [INFO] [AMShutdownThread] |ats.ATSV15HistoryLoggingService|: Stopping ATSService, eventQueueBacklog=0 2017-12-11 19:03:37,090 [INFO] [DelayedContainerManager] |rm.YarnTaskSchedulerService|: AllocatedContainerManager Thread interrupted 2017-12-11 19:03:37,093 [INFO] [AMShutdownThread] |rm.YarnTaskSchedulerService|: Unregistering application from RM, exitStatus=FAILED, exitMessage=Incompatible versions found, clientVersion=0.8.5, AMVersion=0.7.0.2.4.2.0-258 Session stats:submittedDAGs=0, successfulDAGs=0, failedDAGs=0, killedDAGs=0 , trackingURL=http://schedule.will.com:8181/tez-ui-0.8.5?viewPath=%2F%23%2Ftez-app%2Fapplication_1511064572746_31935 2017-12-11 19:03:37,100 [INFO] [AMShutdownThread] |impl.AMRMClientImpl|: Waiting for application to be successfully unregistered. 2017-12-11 19:03:37,202 [INFO] [AMShutdownThread] |rm.YarnTaskSchedulerService|: Successfully unregistered application from RM 2017-12-11 19:03:37,204 [INFO] [AMRM Callback Handler Thread] |impl.AMRMClientAsyncImpl|: Interrupted while waiting for queue java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:287) 2017-12-11 19:03:37,210 [INFO] [AMShutdownThread] |mortbay.log|: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:0 2017-12-11 19:03:37,311 [INFO] [AMShutdownThread] |ipc.Server|: Stopping server on 38848 2017-12-11 19:03:37,312 [INFO] [IPC Server listener on 38848] |ipc.Server|: Stopping IPC Server listener on 38848 2017-12-11 19:03:37,312 [INFO] [AMShutdownThread] |ipc.Server|: Stopping server on 34543 2017-12-11 19:03:37,312 [INFO] [IPC Server Responder] |ipc.Server|: Stopping IPC Server Responder 2017-12-11 19:03:37,312 [INFO] [IPC Server listener on 34543] |ipc.Server|: Stopping IPC Server listener on 34543 2017-12-11 19:03:37,312 [INFO] [IPC Server Responder] |ipc.Server|: Stopping IPC Server Responder 2017-12-11 19:03:37,324 [WARN] [AMShutdownThread] |app.DAGAppMaster|: Failed to delete tez scratch data dir, path=hdfs://dd/tmp/hive2/root/_tez_session_dir/656ffd05-b7b9-433c-9fa8-b8655dc9f840/.tez/application_1511064572746_31935 2017-12-11 19:03:37,324 [INFO] [AMShutdownThread] |app.DAGAppMaster|: Exiting DAGAppMaster..GoodBye! 2017-12-11 19:03:37,326 [INFO] [Thread-3] |app.DAGAppMaster|: DAGAppMasterShutdownHook invoked End of LogType:syslog I am wondering that Ambari apps conflict with apache apps. If do conflict, how could I upgrade my hive to hive2 smoothly, there is a lot of apps running on hive 1.2.1 now. Anybody help me, please. ... View more