Container: container_e02_1496837077102_0003_01_000002 on hdfs-hadoop-0q1h.c.creditvidya-152512.internal_45454_1496852894905 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:directory.info Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:36075 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 84 Jun 7 12:11 container_tokens -rwx------ 1 yarn hadoop 678 Jun 7 12:11 default_container_executor_session.sh -rwx------ 1 yarn hadoop 732 Jun 7 12:11 default_container_executor.sh -rwx------ 1 yarn hadoop 6638 Jun 7 12:11 launch_container.sh lrwxrwxrwx 1 yarn hadoop 71 Jun 7 12:11 py4j-0.10.4-src.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 63 Jun 7 12:11 pyspark.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip lrwxrwxrwx 1 yarn hadoop 77 Jun 7 12:11 __spark_conf__ -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 62 Jun 7 12:11 __spark_libs__ -> /hadoop/yarn/local/filecache/12/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 69 Jun 7 12:11 sparkr -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 12:11 tmp find -L . -maxdepth 5 -ls: 80478612 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 12:11 . 130417841 4 drwx------ 3 yarn hadoop 4096 Jun 7 12:11 ./sparkr 130417842 4 drwx------ 9 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR 130417844 12 -r-x------ 1 yarn hadoop 10818 Jun 7 12:11 ./sparkr/SparkR/NAMESPACE 130417843 4 -r-x------ 1 yarn hadoop 1431 Jun 7 12:11 ./sparkr/SparkR/DESCRIPTION 130417876 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/Meta 130417877 4 -r-x------ 1 yarn hadoop 164 Jun 7 12:11 ./sparkr/SparkR/Meta/Rd.rds 130417880 4 -r-x------ 1 yarn hadoop 38 Jun 7 12:11 ./sparkr/SparkR/Meta/links.rds 130417878 4 -r-x------ 1 yarn hadoop 188 Jun 7 12:11 ./sparkr/SparkR/Meta/hsearch.rds 130417879 4 -r-x------ 1 yarn hadoop 1083 Jun 7 12:11 ./sparkr/SparkR/Meta/package.rds 130417881 4 -r-x------ 1 yarn hadoop 2467 Jun 7 12:11 ./sparkr/SparkR/Meta/nsInfo.rds 130417845 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/worker 130417846 4 -r-x------ 1 yarn hadoop 1975 Jun 7 12:11 ./sparkr/SparkR/worker/daemon.R 130417847 12 -r-x------ 1 yarn hadoop 8950 Jun 7 12:11 ./sparkr/SparkR/worker/worker.R 130417851 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/html 130417852 4 -r-x------ 1 yarn hadoop 1319 Jun 7 12:11 ./sparkr/SparkR/html/R.css 130417853 4 -r-x------ 1 yarn hadoop 937 Jun 7 12:11 ./sparkr/SparkR/html/00Index.html 130417854 4 drwx------ 3 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/tests 130417855 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/tests/testthat 130417873 4 -r-x------ 1 yarn hadoop 1763 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_sparkR.R 130417875 4 -r-x------ 1 yarn hadoop 2388 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_Serde.R 130417871 4 -r-x------ 1 yarn hadoop 1084 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_Windows.R 130417872 4 -r-x------ 1 yarn hadoop 1971 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_includePackage.R 130417856 4 -r-x------ 1 yarn hadoop 1842 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_client.R 130417866 4 -r-x------ 1 yarn hadoop 1332 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 130417860 4 -r-x------ 1 yarn hadoop 1772 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_broadcast.R 130417858 48 -r-x------ 1 yarn hadoop 47315 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_mllib.R 130417859 8 -r-x------ 1 yarn hadoop 4574 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 130417861 108 -r-x------ 1 yarn hadoop 109097 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 130417867 28 -r-x------ 1 yarn hadoop 28197 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_rdd.R 130417868 8 -r-x------ 1 yarn hadoop 8088 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_context.R 130417863 4 -r-x------ 1 yarn hadoop 3231 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 130417874 12 -r-x------ 1 yarn hadoop 9110 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_utils.R 130417865 8 -r-x------ 1 yarn hadoop 5331 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_textFile.R 130417870 4 -r-x------ 1 yarn hadoop 1272 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/jarTest.R 130417862 4 -r-x------ 1 yarn hadoop 3372 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_take.R 130417869 4 -r-x------ 1 yarn hadoop 1001 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 130417864 4 -r-x------ 1 yarn hadoop 3873 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_binary_function.R 130417857 8 -r-x------ 1 yarn hadoop 8128 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_shuffle.R 130417888 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/R 130417889 976 -r-x------ 1 yarn hadoop 999203 Jun 7 12:11 ./sparkr/SparkR/R/SparkR.rdb 130417890 24 -r-x------ 1 yarn hadoop 23836 Jun 7 12:11 ./sparkr/SparkR/R/SparkR.rdx 130417891 4 -r-x------ 1 yarn hadoop 1056 Jun 7 12:11 ./sparkr/SparkR/R/SparkR 130417848 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/profile 130417849 4 -r-x------ 1 yarn hadoop 958 Jun 7 12:11 ./sparkr/SparkR/profile/general.R 130417850 4 -r-x------ 1 yarn hadoop 1715 Jun 7 12:11 ./sparkr/SparkR/profile/shell.R 130417882 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/help 130417887 4 -r-x------ 1 yarn hadoop 60 Jun 7 12:11 ./sparkr/SparkR/help/aliases.rds 130417883 0 -r-x------ 1 yarn hadoop 0 Jun 7 12:11 ./sparkr/SparkR/help/SparkR.rdb 130417884 4 -r-x------ 1 yarn hadoop 64 Jun 7 12:11 ./sparkr/SparkR/help/paths.rds 130417886 0 -r-x------ 1 yarn hadoop 0 Jun 7 12:11 ./sparkr/SparkR/help/AnIndex 130417885 4 -r-x------ 1 yarn hadoop 113 Jun 7 12:11 ./sparkr/SparkR/help/SparkR.rdx 80478613 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 12:11 ./tmp 80478620 4 -rwx------ 1 yarn hadoop 732 Jun 7 12:11 ./default_container_executor.sh 80478616 8 -rwx------ 1 yarn hadoop 6638 Jun 7 12:11 ./launch_container.sh 130417893 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./__spark_conf__ 130417908 4 -r-x------ 1 yarn hadoop 2135 Jun 7 12:11 ./__spark_conf__/capacity-scheduler.xml 130417912 8 -r-x------ 1 yarn hadoop 5305 Jun 7 12:11 ./__spark_conf__/yarn-env.sh 130417911 4 -r-x------ 1 yarn hadoop 1335 Jun 7 12:11 ./__spark_conf__/configuration.xsl 130417894 8 -r-x------ 1 yarn hadoop 6935 Jun 7 12:11 ./__spark_conf__/mapred-site.xml 130417915 4 -r-x------ 1 yarn hadoop 1069 Jun 7 12:11 ./__spark_conf__/container-executor.cfg 130417901 4 -r-x------ 1 yarn hadoop 2622 Jun 7 12:11 ./__spark_conf__/hadoop-metrics2.properties 130417924 8 -r-x------ 1 yarn hadoop 4956 Jun 7 12:11 ./__spark_conf__/metrics.properties 130417907 4 -r-x------ 1 yarn hadoop 884 Jun 7 12:11 ./__spark_conf__/ssl-client.xml 130417918 4 -r-x------ 1 yarn hadoop 856 Jun 7 12:11 ./__spark_conf__/mapred-env.sh 130417916 8 -r-x------ 1 yarn hadoop 4221 Jun 7 12:11 ./__spark_conf__/task-log4j.properties 130417923 4 -r-x------ 1 yarn hadoop 535 Jun 7 12:11 ./__spark_conf__/topology_mappings.data 130417919 4 -r-x------ 1 yarn hadoop 1602 Jun 7 12:11 ./__spark_conf__/health_check 130417928 8 -r-x------ 1 yarn hadoop 4113 Jun 7 12:11 ./__spark_conf__/mapred-queues.xml.template 130417906 4 -r-x------ 1 yarn hadoop 3348 Jun 7 12:11 ./__spark_conf__/core-site.xml 130417909 4 -r-x------ 1 yarn hadoop 758 Jun 7 12:11 ./__spark_conf__/mapred-site.xml.template 130417914 4 -r-x------ 1 yarn hadoop 1020 Jun 7 12:11 ./__spark_conf__/commons-logging.properties 130417905 20 -r-x------ 1 yarn hadoop 17718 Jun 7 12:11 ./__spark_conf__/yarn-site.xml 130417922 4 -r-x------ 1 yarn hadoop 283 Jun 7 12:11 ./__spark_conf__/slaves 130417925 4 -r-x------ 1 yarn hadoop 1000 Jun 7 12:11 ./__spark_conf__/ssl-server.xml 130417929 4 -r-x------ 1 yarn hadoop 3920 Jun 7 12:11 ./__spark_conf__/__spark_conf__.properties 130417926 4 -r-x------ 1 yarn hadoop 2697 Jun 7 12:11 ./__spark_conf__/ssl-server.xml.example 130417910 4 -r-x------ 1 yarn hadoop 2358 Jun 7 12:11 ./__spark_conf__/topology_script.py 130417917 4 -r-x------ 1 yarn hadoop 2490 Jun 7 12:11 ./__spark_conf__/hadoop-metrics.properties 130417920 4 -r-x------ 1 yarn hadoop 2316 Jun 7 12:11 ./__spark_conf__/ssl-client.xml.example 130417896 8 -r-x------ 1 yarn hadoop 6369 Jun 7 12:11 ./__spark_conf__/hadoop-env.sh 130417927 4 -r-x------ 1 yarn hadoop 945 Jun 7 12:11 ./__spark_conf__/taskcontroller.cfg 130417921 4 -r-x------ 1 yarn hadoop 1308 Jun 7 12:11 ./__spark_conf__/hadoop-policy.xml 130417913 8 -r-x------ 1 yarn hadoop 6787 Jun 7 12:11 ./__spark_conf__/hdfs-site.xml 130417895 4 -r-x------ 1 yarn hadoop 620 Jun 7 12:11 ./__spark_conf__/log4j.properties 80478619 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 12:11 ./.default_container_executor_session.sh.crc 80478614 4 -rw-r--r-- 1 yarn hadoop 84 Jun 7 12:11 ./container_tokens 80478615 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 12:11 ./.container_tokens.crc 80478621 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 12:11 ./.default_container_executor.sh.crc 80478607 448 -r-x------ 1 yarn hadoop 454814 Jun 7 12:11 ./pyspark.zip 77595159 12 drwxr-xr-x 2 yarn hadoop 12288 Jun 7 12:11 ./__spark_libs__ 77596121 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 77595901 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 77595824 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 77596170 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 77596156 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 77596149 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 77596186 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 77596145 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 77595162 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 77596147 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 77595765 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 77595910 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 77596188 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 77595175 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 77595926 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 77595919 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 77596173 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 77595807 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 77595816 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 77595188 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 77595828 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 77595780 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 77595811 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 77595829 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 77595179 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 77595818 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 77595908 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 77596129 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 77595806 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 77595776 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 77595835 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 77595165 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 77596015 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 77595774 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 77596124 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 77596050 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 77596160 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 77596157 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 77595771 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 77595928 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 77596123 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 77595931 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 77596152 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 77596162 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 77596179 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 77595914 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 77596000 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 77595790 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 77596138 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 77595817 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 77596139 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 77595782 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 77595796 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 77595927 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 77596120 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 77595812 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 77596018 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 77596144 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 77595834 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 77596003 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 77595916 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 77595988 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 77596017 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 77596141 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 77596174 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 77595832 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 77595923 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 77595786 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 77595990 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 77596155 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 77595792 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 77595799 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 77595778 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 77595922 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 77595784 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 77595813 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 77596159 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 77595160 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 77595789 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 77596154 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 77596161 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 77595797 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 77596182 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 77596014 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 77596110 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 77596178 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 77595904 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 77595961 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 77595766 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 77595917 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 77595815 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 77595168 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 77595801 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 77596116 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 77595800 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 77596135 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 77595794 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 77595925 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 77595820 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 77596163 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 77596189 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 77595911 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 77595809 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 77595182 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 77595821 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 77595808 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 77595167 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 77596114 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 77595805 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 77595810 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 77595905 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 77595798 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 77595989 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 77596010 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 77595772 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 77595907 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 77595791 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 77595767 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 77595902 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 77596131 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 77596035 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 77596153 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 77595773 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 77596112 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 77595903 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 77596136 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 77596118 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 77595783 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 77595918 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 77595163 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 77595770 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 77595785 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 77596180 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 77596175 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 77595900 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 77596128 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 77595187 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 77595830 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 77596137 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 77596146 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 77596132 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 77595869 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 77595183 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 77596125 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 77596122 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 77595788 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 77595924 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 77595164 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 77596187 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 77595804 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 77596167 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 77595777 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 77596028 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 77595802 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 77596151 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 77596142 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 77596134 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 77596183 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 77596169 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 77596140 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 77595768 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 77595831 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 77595912 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 77596172 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 77595793 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 77596177 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 77596115 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 77596127 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 77596113 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 77596143 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 77595795 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 77596184 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 77595921 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 77596171 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 77596150 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 77595181 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 77595833 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 77596117 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 77595781 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 77595920 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 77595803 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 77595814 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 77595775 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 77595913 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 77596004 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 77595825 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 77596119 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 77596176 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 77596126 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 77595161 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 77596190 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 77596166 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 77596111 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 77595929 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 77596020 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 77596148 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 77595915 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 77595176 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 77595174 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 77596133 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 77595787 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 77596165 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 77596158 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 77596185 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 77595819 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 77596168 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 77596164 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 77595173 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 77595172 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 77595769 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 77596034 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 77596012 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 77596130 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 77596023 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 77595909 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 77595906 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 77596181 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 77595996 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 77595171 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 77595169 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 77595779 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 77595822 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 77595930 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 77595933 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 80478610 76 -r-x------ 1 yarn hadoop 74096 Jun 7 12:11 ./py4j-0.10.4-src.zip 80478617 4 -rw-r--r-- 1 yarn hadoop 60 Jun 7 12:11 ./.launch_container.sh.crc 80478618 4 -rwx------ 1 yarn hadoop 678 Jun 7 12:11 ./default_container_executor_session.sh broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:6638 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export SPARK_LOG_URL_STDOUT="http://hdfs-hadoop-0q1h.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000002/zeppelin/stdout?start=-4096" export NM_HOST="hdfs-hadoop-0q1h.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export SPARK_LOG_URL_STDERR="http://hdfs-hadoop-0q1h.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000002/zeppelin/stderr?start=-4096" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000002" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/filecache/12/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/directory.info" exec /bin/bash -c "LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" $JAVA_HOME/bin/java -server -Xmx1024m -Djava.io.tmpdir=$PWD/tmp '-Dspark.history.ui.port=18081' '-Dspark.driver.port=45013' -Dspark.yarn.app.container.log.dir=/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.141.0.9:45013 --executor-id 1 --hostname hdfs-hadoop-0q1h.c.creditvidya-152512.internal --cores 1 --app-id application_1496837077102_0003 --user-class-path file:$PWD/__app__.jar 1>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/stdout 2>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000002/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh LogType:stderr Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:4503 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/yarn/local/filecache/12/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 12:11:34 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 10255@hdfs-hadoop-0q1h 17/06/07 12:11:34 INFO SignalUtils: Registered signal handler for TERM 17/06/07 12:11:34 INFO SignalUtils: Registered signal handler for HUP 17/06/07 12:11:34 INFO SignalUtils: Registered signal handler for INT 17/06/07 12:11:35 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 12:11:35 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 12:11:35 INFO SecurityManager: Changing view acls groups to: 17/06/07 12:11:35 INFO SecurityManager: Changing modify acls groups to: 17/06/07 12:11:35 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 12:11:35 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 71 ms (0 ms spent in bootstraps) 17/06/07 12:11:35 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 12:11:35 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 12:11:35 INFO SecurityManager: Changing view acls groups to: 17/06/07 12:11:35 INFO SecurityManager: Changing modify acls groups to: 17/06/07 12:11:35 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 12:11:35 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 2 ms (0 ms spent in bootstraps) 17/06/07 12:11:36 INFO DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-17555821-6694-43f9-9825-3f59730cd004 17/06/07 12:11:36 INFO DiskBlockManager: Created local directory at /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-51afd0fd-626c-411b-a585-d55ca127c353 17/06/07 12:11:36 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 17/06/07 12:11:36 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.141.0.9:45013 17/06/07 12:11:36 INFO CoarseGrainedExecutorBackend: Successfully registered with driver 17/06/07 12:11:36 INFO Executor: Starting executor ID 1 on host hdfs-hadoop-0q1h.c.creditvidya-152512.internal 17/06/07 12:11:36 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 41824. 17/06/07 12:11:36 INFO NettyBlockTransferService: Server created on hdfs-hadoop-0q1h.c.creditvidya-152512.internal:41824 17/06/07 12:11:36 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/06/07 12:11:36 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(1, hdfs-hadoop-0q1h.c.creditvidya-152512.internal, 41824, None) 17/06/07 12:11:36 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(1, hdfs-hadoop-0q1h.c.creditvidya-152512.internal, 41824, None) 17/06/07 12:11:36 INFO BlockManager: external shuffle service port = 7447 17/06/07 12:11:36 INFO BlockManager: Registering executor with local external shuffle service. 17/06/07 12:11:36 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-0q1h.c.creditvidya-152512.internal/10.141.0.5:7447 after 1 ms (0 ms spent in bootstraps) 17/06/07 12:11:36 INFO BlockManager: Initialized BlockManager: BlockManagerId(1, hdfs-hadoop-0q1h.c.creditvidya-152512.internal, 41824, None) 17/06/07 12:11:36 INFO Executor: Using REPL class URI: spark://10.141.0.9:45013/classes 17/06/07 12:12:37 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/06/07 12:12:37 INFO DiskBlockManager: Shutdown hook called 17/06/07 12:12:37 INFO ShutdownHookManager: Shutdown hook called End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout Container: container_e02_1496837077102_0003_01_000001 on hdfs-hadoop-0q1h.c.creditvidya-152512.internal_45454_1496852894905 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:directory.info Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:36085 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 69 Jun 7 12:11 container_tokens -rwx------ 1 yarn hadoop 685 Jun 7 12:11 default_container_executor_session.sh -rwx------ 1 yarn hadoop 739 Jun 7 12:11 default_container_executor.sh -rwx------ 1 yarn hadoop 6230 Jun 7 12:11 launch_container.sh lrwxrwxrwx 1 yarn hadoop 71 Jun 7 12:11 py4j-0.10.4-src.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 63 Jun 7 12:11 pyspark.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip lrwxrwxrwx 1 yarn hadoop 77 Jun 7 12:11 __spark_conf__ -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 62 Jun 7 12:11 __spark_libs__ -> /hadoop/yarn/local/filecache/12/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 69 Jun 7 12:11 sparkr -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 12:11 tmp find -L . -maxdepth 5 -ls: 130417837 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 12:11 . 130417841 4 drwx------ 3 yarn hadoop 4096 Jun 7 12:11 ./sparkr 130417842 4 drwx------ 9 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR 130417844 12 -r-x------ 1 yarn hadoop 10818 Jun 7 12:11 ./sparkr/SparkR/NAMESPACE 130417843 4 -r-x------ 1 yarn hadoop 1431 Jun 7 12:11 ./sparkr/SparkR/DESCRIPTION 130417876 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/Meta 130417877 4 -r-x------ 1 yarn hadoop 164 Jun 7 12:11 ./sparkr/SparkR/Meta/Rd.rds 130417880 4 -r-x------ 1 yarn hadoop 38 Jun 7 12:11 ./sparkr/SparkR/Meta/links.rds 130417878 4 -r-x------ 1 yarn hadoop 188 Jun 7 12:11 ./sparkr/SparkR/Meta/hsearch.rds 130417879 4 -r-x------ 1 yarn hadoop 1083 Jun 7 12:11 ./sparkr/SparkR/Meta/package.rds 130417881 4 -r-x------ 1 yarn hadoop 2467 Jun 7 12:11 ./sparkr/SparkR/Meta/nsInfo.rds 130417845 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/worker 130417846 4 -r-x------ 1 yarn hadoop 1975 Jun 7 12:11 ./sparkr/SparkR/worker/daemon.R 130417847 12 -r-x------ 1 yarn hadoop 8950 Jun 7 12:11 ./sparkr/SparkR/worker/worker.R 130417851 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/html 130417852 4 -r-x------ 1 yarn hadoop 1319 Jun 7 12:11 ./sparkr/SparkR/html/R.css 130417853 4 -r-x------ 1 yarn hadoop 937 Jun 7 12:11 ./sparkr/SparkR/html/00Index.html 130417854 4 drwx------ 3 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/tests 130417855 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/tests/testthat 130417873 4 -r-x------ 1 yarn hadoop 1763 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_sparkR.R 130417875 4 -r-x------ 1 yarn hadoop 2388 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_Serde.R 130417871 4 -r-x------ 1 yarn hadoop 1084 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_Windows.R 130417872 4 -r-x------ 1 yarn hadoop 1971 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_includePackage.R 130417856 4 -r-x------ 1 yarn hadoop 1842 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_client.R 130417866 4 -r-x------ 1 yarn hadoop 1332 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 130417860 4 -r-x------ 1 yarn hadoop 1772 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_broadcast.R 130417858 48 -r-x------ 1 yarn hadoop 47315 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_mllib.R 130417859 8 -r-x------ 1 yarn hadoop 4574 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 130417861 108 -r-x------ 1 yarn hadoop 109097 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 130417867 28 -r-x------ 1 yarn hadoop 28197 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_rdd.R 130417868 8 -r-x------ 1 yarn hadoop 8088 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_context.R 130417863 4 -r-x------ 1 yarn hadoop 3231 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 130417874 12 -r-x------ 1 yarn hadoop 9110 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_utils.R 130417865 8 -r-x------ 1 yarn hadoop 5331 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_textFile.R 130417870 4 -r-x------ 1 yarn hadoop 1272 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/jarTest.R 130417862 4 -r-x------ 1 yarn hadoop 3372 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_take.R 130417869 4 -r-x------ 1 yarn hadoop 1001 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 130417864 4 -r-x------ 1 yarn hadoop 3873 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_binary_function.R 130417857 8 -r-x------ 1 yarn hadoop 8128 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_shuffle.R 130417888 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/R 130417889 976 -r-x------ 1 yarn hadoop 999203 Jun 7 12:11 ./sparkr/SparkR/R/SparkR.rdb 130417890 24 -r-x------ 1 yarn hadoop 23836 Jun 7 12:11 ./sparkr/SparkR/R/SparkR.rdx 130417891 4 -r-x------ 1 yarn hadoop 1056 Jun 7 12:11 ./sparkr/SparkR/R/SparkR 130417848 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/profile 130417849 4 -r-x------ 1 yarn hadoop 958 Jun 7 12:11 ./sparkr/SparkR/profile/general.R 130417850 4 -r-x------ 1 yarn hadoop 1715 Jun 7 12:11 ./sparkr/SparkR/profile/shell.R 130417882 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/help 130417887 4 -r-x------ 1 yarn hadoop 60 Jun 7 12:11 ./sparkr/SparkR/help/aliases.rds 130417883 0 -r-x------ 1 yarn hadoop 0 Jun 7 12:11 ./sparkr/SparkR/help/SparkR.rdb 130417884 4 -r-x------ 1 yarn hadoop 64 Jun 7 12:11 ./sparkr/SparkR/help/paths.rds 130417886 0 -r-x------ 1 yarn hadoop 0 Jun 7 12:11 ./sparkr/SparkR/help/AnIndex 130417885 4 -r-x------ 1 yarn hadoop 113 Jun 7 12:11 ./sparkr/SparkR/help/SparkR.rdx 130417840 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 12:11 ./tmp 130417935 4 -rwx------ 1 yarn hadoop 739 Jun 7 12:11 ./default_container_executor.sh 130417931 8 -rwx------ 1 yarn hadoop 6230 Jun 7 12:11 ./launch_container.sh 130417893 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./__spark_conf__ 130417908 4 -r-x------ 1 yarn hadoop 2135 Jun 7 12:11 ./__spark_conf__/capacity-scheduler.xml 130417912 8 -r-x------ 1 yarn hadoop 5305 Jun 7 12:11 ./__spark_conf__/yarn-env.sh 130417911 4 -r-x------ 1 yarn hadoop 1335 Jun 7 12:11 ./__spark_conf__/configuration.xsl 130417894 8 -r-x------ 1 yarn hadoop 6935 Jun 7 12:11 ./__spark_conf__/mapred-site.xml 130417915 4 -r-x------ 1 yarn hadoop 1069 Jun 7 12:11 ./__spark_conf__/container-executor.cfg 130417901 4 -r-x------ 1 yarn hadoop 2622 Jun 7 12:11 ./__spark_conf__/hadoop-metrics2.properties 130417924 8 -r-x------ 1 yarn hadoop 4956 Jun 7 12:11 ./__spark_conf__/metrics.properties 130417907 4 -r-x------ 1 yarn hadoop 884 Jun 7 12:11 ./__spark_conf__/ssl-client.xml 130417918 4 -r-x------ 1 yarn hadoop 856 Jun 7 12:11 ./__spark_conf__/mapred-env.sh 130417916 8 -r-x------ 1 yarn hadoop 4221 Jun 7 12:11 ./__spark_conf__/task-log4j.properties 130417923 4 -r-x------ 1 yarn hadoop 535 Jun 7 12:11 ./__spark_conf__/topology_mappings.data 130417919 4 -r-x------ 1 yarn hadoop 1602 Jun 7 12:11 ./__spark_conf__/health_check 130417928 8 -r-x------ 1 yarn hadoop 4113 Jun 7 12:11 ./__spark_conf__/mapred-queues.xml.template 130417906 4 -r-x------ 1 yarn hadoop 3348 Jun 7 12:11 ./__spark_conf__/core-site.xml 130417909 4 -r-x------ 1 yarn hadoop 758 Jun 7 12:11 ./__spark_conf__/mapred-site.xml.template 130417914 4 -r-x------ 1 yarn hadoop 1020 Jun 7 12:11 ./__spark_conf__/commons-logging.properties 130417905 20 -r-x------ 1 yarn hadoop 17718 Jun 7 12:11 ./__spark_conf__/yarn-site.xml 130417922 4 -r-x------ 1 yarn hadoop 283 Jun 7 12:11 ./__spark_conf__/slaves 130417925 4 -r-x------ 1 yarn hadoop 1000 Jun 7 12:11 ./__spark_conf__/ssl-server.xml 130417929 4 -r-x------ 1 yarn hadoop 3920 Jun 7 12:11 ./__spark_conf__/__spark_conf__.properties 130417926 4 -r-x------ 1 yarn hadoop 2697 Jun 7 12:11 ./__spark_conf__/ssl-server.xml.example 130417910 4 -r-x------ 1 yarn hadoop 2358 Jun 7 12:11 ./__spark_conf__/topology_script.py 130417917 4 -r-x------ 1 yarn hadoop 2490 Jun 7 12:11 ./__spark_conf__/hadoop-metrics.properties 130417920 4 -r-x------ 1 yarn hadoop 2316 Jun 7 12:11 ./__spark_conf__/ssl-client.xml.example 130417896 8 -r-x------ 1 yarn hadoop 6369 Jun 7 12:11 ./__spark_conf__/hadoop-env.sh 130417927 4 -r-x------ 1 yarn hadoop 945 Jun 7 12:11 ./__spark_conf__/taskcontroller.cfg 130417921 4 -r-x------ 1 yarn hadoop 1308 Jun 7 12:11 ./__spark_conf__/hadoop-policy.xml 130417913 8 -r-x------ 1 yarn hadoop 6787 Jun 7 12:11 ./__spark_conf__/hdfs-site.xml 130417895 4 -r-x------ 1 yarn hadoop 620 Jun 7 12:11 ./__spark_conf__/log4j.properties 130417934 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 12:11 ./.default_container_executor_session.sh.crc 130417892 4 -rw-r--r-- 1 yarn hadoop 69 Jun 7 12:11 ./container_tokens 130417930 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 12:11 ./.container_tokens.crc 130417936 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 12:11 ./.default_container_executor.sh.crc 80478607 448 -r-x------ 1 yarn hadoop 454814 Jun 7 12:11 ./pyspark.zip 77595159 12 drwxr-xr-x 2 yarn hadoop 12288 Jun 7 12:11 ./__spark_libs__ 77596121 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 77595901 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 77595824 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 77596170 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 77596156 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 77596149 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 77596186 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 77596145 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 77595162 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 77596147 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 77595765 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 77595910 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 77596188 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 77595175 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 77595926 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 77595919 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 77596173 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 77595807 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 77595816 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 77595188 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 77595828 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 77595780 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 77595811 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 77595829 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 77595179 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 77595818 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 77595908 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 77596129 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 77595806 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 77595776 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 77595835 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 77595165 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 77596015 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 77595774 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 77596124 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 77596050 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 77596160 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 77596157 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 77595771 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 77595928 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 77596123 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 77595931 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 77596152 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 77596162 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 77596179 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 77595914 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 77596000 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 77595790 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 77596138 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 77595817 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 77596139 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 77595782 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 77595796 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 77595927 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 77596120 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 77595812 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 77596018 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 77596144 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 77595834 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 77596003 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 77595916 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 77595988 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 77596017 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 77596141 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 77596174 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 77595832 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 77595923 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 77595786 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 77595990 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 77596155 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 77595792 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 77595799 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 77595778 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 77595922 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 77595784 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 77595813 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 77596159 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 77595160 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 77595789 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 77596154 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 77596161 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 77595797 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 77596182 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 77596014 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 77596110 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 77596178 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 77595904 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 77595961 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 77595766 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 77595917 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 77595815 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 77595168 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 77595801 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 77596116 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 77595800 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 77596135 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 77595794 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 77595925 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 77595820 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 77596163 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 77596189 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 77595911 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 77595809 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 77595182 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 77595821 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 77595808 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 77595167 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 77596114 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 77595805 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 77595810 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 77595905 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 77595798 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 77595989 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 77596010 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 77595772 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 77595907 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 77595791 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 77595767 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 77595902 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 77596131 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 77596035 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 77596153 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 77595773 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 77596112 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 77595903 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 77596136 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 77596118 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 77595783 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 77595918 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 77595163 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 77595770 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 77595785 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 77596180 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 77596175 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 77595900 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 77596128 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 77595187 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 77595830 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 77596137 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 77596146 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 77596132 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 77595869 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 77595183 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 77596125 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 77596122 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 77595788 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 77595924 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 77595164 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 77596187 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 77595804 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 77596167 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 77595777 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 77596028 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 77595802 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 77596151 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 77596142 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 77596134 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 77596183 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 77596169 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 77596140 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 77595768 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 77595831 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 77595912 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 77596172 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 77595793 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 77596177 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 77596115 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 77596127 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 77596113 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 77596143 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 77595795 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 77596184 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 77595921 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 77596171 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 77596150 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 77595181 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 77595833 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 77596117 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 77595781 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 77595920 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 77595803 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 77595814 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 77595775 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 77595913 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 77596004 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 77595825 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 77596119 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 77596176 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 77596126 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 77595161 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 77596190 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 77596166 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 77596111 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 77595929 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 77596020 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 77596148 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 77595915 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 77595176 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 77595174 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 77596133 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 77595787 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 77596165 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 77596158 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 77596185 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 77595819 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 77596168 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 77596164 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 77595173 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 77595172 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 77595769 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 77596034 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 77596012 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 77596130 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 77596023 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 77595909 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 77595906 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 77596181 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 77595996 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 77595171 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 77595169 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 77595779 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 77595822 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 77595930 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 77595933 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 80478610 76 -r-x------ 1 yarn hadoop 74096 Jun 7 12:11 ./py4j-0.10.4-src.zip 130417932 4 -rw-r--r-- 1 yarn hadoop 60 Jun 7 12:11 ./.launch_container.sh.crc 130417933 4 -rwx------ 1 yarn hadoop 685 Jun 7 12:11 ./default_container_executor_session.sh broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:6230 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export MAX_APP_ATTEMPTS="2" export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export APP_SUBMIT_TIME_ENV="1496837487051" export NM_HOST="hdfs-hadoop-0q1h.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip" export APPLICATION_WEB_PROXY_BASE="/proxy/application_1496837077102_0003" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="/etc/zeppelin/conf/external-dependency-conf::/usr/hdp/current/zeppelin-server/interpreter/spark/*:/usr/hdp/current/zeppelin-server/lib/interpreter/*::/usr/hdp/current/zeppelin-server/interpreter/spark/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar:$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000001" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/filecache/12/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/directory.info" exec /bin/bash -c "$JAVA_HOME/bin/java -server -Xmx512m -Djava.io.tmpdir=$PWD/tmp -Dhdp.version=2.6.0.3-8 -Dspark.yarn.app.container.log.dir=/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001 org.apache.spark.deploy.yarn.ExecutorLauncher --arg '10.141.0.9:45013' --properties-file $PWD/__spark_conf__/__spark_conf__.properties 1> /hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/stdout 2> /hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000001/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh LogType:stderr Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:13960 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/yarn/local/filecache/12/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 12:11:30 INFO SignalUtils: Registered signal handler for TERM 17/06/07 12:11:30 INFO SignalUtils: Registered signal handler for HUP 17/06/07 12:11:30 INFO SignalUtils: Registered signal handler for INT 17/06/07 12:11:31 INFO ApplicationMaster: Preparing Local resources 17/06/07 12:11:32 INFO ApplicationMaster: ApplicationAttemptId: appattempt_1496837077102_0003_000001 17/06/07 12:11:32 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 12:11:32 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 12:11:32 INFO SecurityManager: Changing view acls groups to: 17/06/07 12:11:32 INFO SecurityManager: Changing modify acls groups to: 17/06/07 12:11:32 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 12:11:32 INFO ApplicationMaster: Waiting for Spark driver to be reachable. 17/06/07 12:11:32 INFO ApplicationMaster: Driver now available: 10.141.0.9:45013 17/06/07 12:11:32 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 59 ms (0 ms spent in bootstraps) 17/06/07 12:11:32 INFO ApplicationMaster$AMEndpoint: Add WebUI Filter. AddWebUIFilter(org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter,Map(PROXY_HOSTS -> hdfs-hadoop-88hh.c.creditvidya-152512.internal, PROXY_URI_BASES -> http://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8088/proxy/application_1496837077102_0003),/proxy/application_1496837077102_0003) 17/06/07 12:11:32 INFO ApplicationMaster: =============================================================================== YARN executor launch context: env: SPARK_YARN_USER_ENV -> PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip CLASSPATH -> {{PWD}}{{PWD}}/__spark_conf__{{PWD}}/__spark_libs__/*/etc/hadoop/conf/usr/hdp/current/hadoop-client/*/usr/hdp/current/hadoop-client/lib/*/usr/hdp/current/hadoop-hdfs-client/*/usr/hdp/current/hadoop-hdfs-client/lib/*/usr/hdp/current/hadoop-yarn-client/*/usr/hdp/current/hadoop-yarn-client/lib/*$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure SPARK_YARN_STAGING_DIR -> hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003 SPARK_USER -> zeppelin SPARK_YARN_MODE -> true PYTHONPATH -> /usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip{{PWD}}/pyspark.zip{{PWD}}/py4j-0.10.4-src.zip{{PWD}}/pyspark.zip{{PWD}}/py4j-0.10.4-src.zip/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip command: LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" \ {{JAVA_HOME}}/bin/java \ -server \ -Xmx1024m \ -Djava.io.tmpdir={{PWD}}/tmp \ '-Dspark.history.ui.port=18081' \ '-Dspark.driver.port=45013' \ -Dspark.yarn.app.container.log.dir= \ -XX:OnOutOfMemoryError='kill %p' \ org.apache.spark.executor.CoarseGrainedExecutorBackend \ --driver-url \ spark://CoarseGrainedScheduler@10.141.0.9:45013 \ --executor-id \ \ --hostname \ \ --cores \ 1 \ --app-id \ application_1496837077102_0003 \ --user-class-path \ file:$PWD/__app__.jar \ 1>/stdout \ 2>/stderr resources: py4j-0.10.4-src.zip -> resource { scheme: "hdfs" host: "hdfs-hadoop-88hh.c.creditvidya-152512.internal" port: 8020 file: "/user/zeppelin/.sparkStaging/application_1496837077102_0003/py4j-0.10.4-src.zip" } size: 74096 timestamp: 1496837486850 type: FILE visibility: PRIVATE __spark_conf__ -> resource { scheme: "hdfs" host: "hdfs-hadoop-88hh.c.creditvidya-152512.internal" port: 8020 file: "/user/zeppelin/.sparkStaging/application_1496837077102_0003/__spark_conf__.zip" } size: 94388 timestamp: 1496837486997 type: ARCHIVE visibility: PRIVATE sparkr -> resource { scheme: "hdfs" host: "hdfs-hadoop-88hh.c.creditvidya-152512.internal" port: 8020 file: "/user/zeppelin/.sparkStaging/application_1496837077102_0003/sparkr.zip" } size: 681589 timestamp: 1496837486905 type: ARCHIVE visibility: PRIVATE pyspark.zip -> resource { scheme: "hdfs" host: "hdfs-hadoop-88hh.c.creditvidya-152512.internal" port: 8020 file: "/user/zeppelin/.sparkStaging/application_1496837077102_0003/pyspark.zip" } size: 454814 timestamp: 1496837486796 type: FILE visibility: PRIVATE __spark_libs__ -> resource { scheme: "hdfs" host: "hdfs-hadoop-88hh.c.creditvidya-152512.internal" port: 8020 file: "/hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz" } size: 180333451 timestamp: 1496837311912 type: ARCHIVE visibility: PUBLIC =============================================================================== 17/06/07 12:11:32 INFO RMProxy: Connecting to ResourceManager at hdfs-hadoop-88hh.c.creditvidya-152512.internal/10.141.0.2:8030 17/06/07 12:11:32 INFO YarnRMClient: Registering the ApplicationMaster 17/06/07 12:11:32 INFO Utils: Using initial executors = 2, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances 17/06/07 12:11:32 INFO YarnAllocator: Will request 2 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead) 17/06/07 12:11:32 INFO YarnAllocator: Submitted 2 unlocalized container requests. 17/06/07 12:11:33 INFO ApplicationMaster: Started progress reporter thread with (heartbeat : 3000, initial allocation : 200) intervals 17/06/07 12:11:33 INFO AMRMClientImpl: Received new token for : hdfs-hadoop-0q1h.c.creditvidya-152512.internal:45454 17/06/07 12:11:33 INFO AMRMClientImpl: Received new token for : hdfs-hadoop-p0z7.c.creditvidya-152512.internal:45454 17/06/07 12:11:33 INFO YarnAllocator: Launching container container_e02_1496837077102_0003_01_000002 on host hdfs-hadoop-0q1h.c.creditvidya-152512.internal 17/06/07 12:11:33 INFO YarnAllocator: Launching container container_e02_1496837077102_0003_01_000003 on host hdfs-hadoop-p0z7.c.creditvidya-152512.internal 17/06/07 12:11:33 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 17/06/07 12:11:33 INFO YarnAllocator: Received 2 containers from YARN, launching executors on 2 of them. 17/06/07 12:11:33 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 17/06/07 12:11:33 INFO ContainerManagementProtocolProxy: Opening proxy : hdfs-hadoop-p0z7.c.creditvidya-152512.internal:45454 17/06/07 12:11:33 INFO ContainerManagementProtocolProxy: Opening proxy : hdfs-hadoop-0q1h.c.creditvidya-152512.internal:45454 17/06/07 12:12:36 INFO YarnAllocator: Driver requested a total number of 1 executor(s). 17/06/07 12:12:36 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1. 17/06/07 12:12:36 INFO YarnAllocator: Driver requested a total number of 0 executor(s). 17/06/07 12:12:36 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 2. 17/06/07 13:10:47 INFO YarnAllocator: Driver requested a total number of 1 executor(s). 17/06/07 13:10:47 INFO YarnAllocator: Will request 1 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead) 17/06/07 13:10:47 INFO YarnAllocator: Submitted container request for host hdfs-hadoop-6rgx.c.creditvidya-152512.internal,hdfs-hadoop-gs1f.c.creditvidya-152512.internal,hdfs-hadoop-mr2w.c.creditvidya-152512.internal,hdfs-hadoop-0j7l.c.creditvidya-152512.internal,hdfs-hadoop-0q1h.c.creditvidya-152512.internal. 17/06/07 13:10:48 INFO YarnAllocator: Driver requested a total number of 3 executor(s). 17/06/07 13:10:48 INFO YarnAllocator: Will request 2 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead) 17/06/07 13:10:48 INFO YarnAllocator: Submitted container request for host hdfs-hadoop-6rgx.c.creditvidya-152512.internal,hdfs-hadoop-gs1f.c.creditvidya-152512.internal,hdfs-hadoop-mr2w.c.creditvidya-152512.internal,hdfs-hadoop-0j7l.c.creditvidya-152512.internal,hdfs-hadoop-0q1h.c.creditvidya-152512.internal. 17/06/07 13:10:48 INFO YarnAllocator: Submitted container request for host hdfs-hadoop-0q1h.c.creditvidya-152512.internal. 17/06/07 13:10:48 INFO AMRMClientImpl: Received new token for : hdfs-hadoop-mr2w.c.creditvidya-152512.internal:45454 17/06/07 13:10:48 INFO YarnAllocator: Launching container container_e02_1496837077102_0003_01_000004 on host hdfs-hadoop-mr2w.c.creditvidya-152512.internal 17/06/07 13:10:48 INFO YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. 17/06/07 13:10:48 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 17/06/07 13:10:48 INFO ContainerManagementProtocolProxy: Opening proxy : hdfs-hadoop-mr2w.c.creditvidya-152512.internal:45454 17/06/07 13:10:48 INFO AMRMClientImpl: Received new token for : hdfs-hadoop-gs1f.c.creditvidya-152512.internal:45454 17/06/07 13:10:48 INFO YarnAllocator: Launching container container_e02_1496837077102_0003_01_000005 on host hdfs-hadoop-gs1f.c.creditvidya-152512.internal 17/06/07 13:10:48 INFO YarnAllocator: Launching container container_e02_1496837077102_0003_01_000006 on host hdfs-hadoop-gs1f.c.creditvidya-152512.internal 17/06/07 13:10:48 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 17/06/07 13:10:48 INFO YarnAllocator: Received 2 containers from YARN, launching executors on 2 of them. 17/06/07 13:10:48 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 17/06/07 13:10:48 INFO ContainerManagementProtocolProxy: Opening proxy : hdfs-hadoop-gs1f.c.creditvidya-152512.internal:45454 17/06/07 13:10:48 INFO ContainerManagementProtocolProxy: Opening proxy : hdfs-hadoop-gs1f.c.creditvidya-152512.internal:45454 17/06/07 13:10:49 INFO YarnAllocator: Driver requested a total number of 4 executor(s). 17/06/07 13:10:49 INFO YarnAllocator: Will request 1 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead) 17/06/07 13:10:49 INFO YarnAllocator: Submitted container request for host hdfs-hadoop-6rgx.c.creditvidya-152512.internal,hdfs-hadoop-0j7l.c.creditvidya-152512.internal,hdfs-hadoop-0q1h.c.creditvidya-152512.internal. 17/06/07 13:10:50 INFO YarnAllocator: Launching container container_e02_1496837077102_0003_01_000007 on host hdfs-hadoop-gs1f.c.creditvidya-152512.internal 17/06/07 13:10:50 INFO YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. 17/06/07 13:10:50 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 17/06/07 13:10:50 INFO ContainerManagementProtocolProxy: Opening proxy : hdfs-hadoop-gs1f.c.creditvidya-152512.internal:45454 17/06/07 13:10:55 INFO YarnAllocator: Driver requested a total number of 3 executor(s). 17/06/07 13:10:55 INFO YarnAllocator: Driver requested a total number of 1 executor(s). 17/06/07 13:10:57 INFO YarnAllocator: Driver requested a total number of 0 executor(s). 17/06/07 13:11:55 INFO YarnAllocator: Driver requested a total number of 3 executor(s). 17/06/07 13:11:55 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 3. 17/06/07 13:11:55 INFO YarnAllocator: Driver requested a total number of 2 executor(s). 17/06/07 13:11:55 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 4. 17/06/07 13:11:56 INFO YarnAllocator: Driver requested a total number of 1 executor(s). 17/06/07 13:11:56 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 5. 17/06/07 13:12:28 INFO YarnAllocator: Driver requested a total number of 0 executor(s). 17/06/07 13:12:28 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 6. 17/06/07 16:28:13 INFO ApplicationMaster$AMEndpoint: Driver terminated or disconnected! Shutting down. 10.141.0.9:45013 17/06/07 16:28:13 INFO ApplicationMaster$AMEndpoint: Driver terminated or disconnected! Shutting down. 10.141.0.9:45013 17/06/07 16:28:13 INFO ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0 17/06/07 16:28:13 INFO ApplicationMaster: Unregistering ApplicationMaster with SUCCEEDED 17/06/07 16:28:13 INFO AMRMClientImpl: Waiting for application to be successfully unregistered. 17/06/07 16:28:13 INFO ApplicationMaster: Deleting staging directory hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003 17/06/07 16:28:13 INFO ShutdownHookManager: Shutdown hook called End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout Container: container_e02_1496837077102_0003_01_000006 on hdfs-hadoop-gs1f.c.creditvidya-152512.internal_45454_1496852894116 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:stderr Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:7959 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 13:10:49 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 2889@hdfs-hadoop-gs1f 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for TERM 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for HUP 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for INT 17/06/07 13:10:50 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:50 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:50 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 65 ms (0 ms spent in bootstraps) 17/06/07 13:10:50 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:50 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:50 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-d2c05dca-cd99-41d6-aa5e-6e13a90969d8 17/06/07 13:10:51 INFO DiskBlockManager: Created local directory at /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-278815f1-a2e6-4aa6-9c0b-f7db89ec4c00 17/06/07 13:10:51 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.141.0.9:45013 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Successfully registered with driver 17/06/07 13:10:51 INFO Executor: Starting executor ID 5 on host hdfs-hadoop-gs1f.c.creditvidya-152512.internal 17/06/07 13:10:51 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 42460. 17/06/07 13:10:51 INFO NettyBlockTransferService: Server created on hdfs-hadoop-gs1f.c.creditvidya-152512.internal:42460 17/06/07 13:10:51 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/06/07 13:10:51 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(5, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42460, None) 17/06/07 13:10:51 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(5, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42460, None) 17/06/07 13:10:51 INFO BlockManager: external shuffle service port = 7447 17/06/07 13:10:51 INFO BlockManager: Registering executor with local external shuffle service. 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-gs1f.c.creditvidya-152512.internal/10.141.0.6:7447 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO BlockManager: Initialized BlockManager: BlockManagerId(5, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42460, None) 17/06/07 13:10:51 INFO Executor: Using REPL class URI: spark://10.141.0.9:45013/classes 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Got assigned task 2 17/06/07 13:10:51 INFO Executor: Running task 2.0 in stage 0.0 (TID 2) 17/06/07 13:10:51 INFO Executor: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar with timestamp 1496837484445 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 3 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO Utils: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-a35dfe60-37f0-40a6-99b8-9a8f8303c10f/fetchFileTemp3332396155740554898.tmp 17/06/07 13:10:51 INFO Utils: Copying /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-a35dfe60-37f0-40a6-99b8-9a8f8303c10f/-2200289241496837484445_cache to /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar 17/06/07 13:10:51 INFO Executor: Adding file:/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to class loader 17/06/07 13:10:51 INFO TorrentBroadcast: Started reading broadcast variable 1 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-gs1f.c.creditvidya-152512.internal/10.141.0.6:42192 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 3.3 KB, free 366.3 MB) 17/06/07 13:10:51 INFO TorrentBroadcast: Reading broadcast variable 1 took 110 ms 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 5.4 KB, free 366.3 MB) 17/06/07 13:10:51 INFO HadoopRDD: Input split: hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/tenants/1/installation/2017/03/l0/sms_data.json:268435456+134217728 17/06/07 13:10:51 INFO TorrentBroadcast: Started reading broadcast variable 0 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 33.0 KB, free 366.3 MB) 17/06/07 13:10:51 INFO TorrentBroadcast: Reading broadcast variable 0 took 12 ms 17/06/07 13:10:52 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 490.9 KB, free 365.8 MB) 17/06/07 13:10:52 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 17/06/07 13:10:52 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 17/06/07 13:10:52 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 17/06/07 13:10:52 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 17/06/07 13:10:52 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 17/06/07 13:10:55 INFO Executor: Finished task 2.0 in stage 0.0 (TID 2). 3766 bytes result sent to driver 17/06/07 13:11:56 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/06/07 13:11:56 INFO DiskBlockManager: Shutdown hook called 17/06/07 13:11:56 INFO ShutdownHookManager: Shutdown hook called 17/06/07 13:11:56 INFO ShutdownHookManager: Deleting directory /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-ee659919-84cc-4593-8b1b-81c7925a9e93 17/06/07 13:11:56 INFO ShutdownHookManager: Deleting directory /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-a35dfe60-37f0-40a6-99b8-9a8f8303c10f End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout LogType:directory.info Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:36082 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 container_tokens -rwx------ 1 yarn hadoop 685 Jun 7 13:10 default_container_executor_session.sh -rwx------ 1 yarn hadoop 739 Jun 7 13:10 default_container_executor.sh -rwx------ 1 yarn hadoop 6680 Jun 7 13:10 launch_container.sh lrwxrwxrwx 1 yarn hadoop 78 Jun 7 13:10 py4j-0.10.4-src.zip -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 70 Jun 7 13:10 pyspark.zip -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip lrwxrwxrwx 1 yarn hadoop 70 Jun 7 13:10 __spark_conf__ -> /hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 69 Jun 7 13:10 __spark_libs__ -> /hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 62 Jun 7 13:10 sparkr -> /hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 tmp find -L . -maxdepth 5 -ls: 70254845 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 13:10 . 70254955 4 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 ./container_tokens 111412505 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./__spark_conf__ 111412507 4 -r-x------ 1 yarn hadoop 620 Jun 7 13:10 ./__spark_conf__/log4j.properties 111412533 8 -r-x------ 1 yarn hadoop 4113 Jun 7 13:10 ./__spark_conf__/mapred-queues.xml.template 111412532 4 -r-x------ 1 yarn hadoop 945 Jun 7 13:10 ./__spark_conf__/taskcontroller.cfg 111412510 20 -r-x------ 1 yarn hadoop 17718 Jun 7 13:10 ./__spark_conf__/yarn-site.xml 111412516 4 -r-x------ 1 yarn hadoop 1335 Jun 7 13:10 ./__spark_conf__/configuration.xsl 111412514 4 -r-x------ 1 yarn hadoop 758 Jun 7 13:10 ./__spark_conf__/mapred-site.xml.template 111412523 4 -r-x------ 1 yarn hadoop 856 Jun 7 13:10 ./__spark_conf__/mapred-env.sh 111412525 4 -r-x------ 1 yarn hadoop 2316 Jun 7 13:10 ./__spark_conf__/ssl-client.xml.example 111412512 4 -r-x------ 1 yarn hadoop 884 Jun 7 13:10 ./__spark_conf__/ssl-client.xml 111412526 4 -r-x------ 1 yarn hadoop 1308 Jun 7 13:10 ./__spark_conf__/hadoop-policy.xml 111412522 4 -r-x------ 1 yarn hadoop 2490 Jun 7 13:10 ./__spark_conf__/hadoop-metrics.properties 111412520 4 -r-x------ 1 yarn hadoop 1069 Jun 7 13:10 ./__spark_conf__/container-executor.cfg 111412530 4 -r-x------ 1 yarn hadoop 1000 Jun 7 13:10 ./__spark_conf__/ssl-server.xml 111412521 8 -r-x------ 1 yarn hadoop 4221 Jun 7 13:10 ./__spark_conf__/task-log4j.properties 111412529 8 -r-x------ 1 yarn hadoop 4956 Jun 7 13:10 ./__spark_conf__/metrics.properties 111412517 8 -r-x------ 1 yarn hadoop 5305 Jun 7 13:10 ./__spark_conf__/yarn-env.sh 111412519 4 -r-x------ 1 yarn hadoop 1020 Jun 7 13:10 ./__spark_conf__/commons-logging.properties 111412509 4 -r-x------ 1 yarn hadoop 2622 Jun 7 13:10 ./__spark_conf__/hadoop-metrics2.properties 111412515 4 -r-x------ 1 yarn hadoop 2358 Jun 7 13:10 ./__spark_conf__/topology_script.py 111412524 4 -r-x------ 1 yarn hadoop 1602 Jun 7 13:10 ./__spark_conf__/health_check 111412527 4 -r-x------ 1 yarn hadoop 283 Jun 7 13:10 ./__spark_conf__/slaves 111412518 8 -r-x------ 1 yarn hadoop 6787 Jun 7 13:10 ./__spark_conf__/hdfs-site.xml 111412534 4 -r-x------ 1 yarn hadoop 3920 Jun 7 13:10 ./__spark_conf__/__spark_conf__.properties 111412513 4 -r-x------ 1 yarn hadoop 2135 Jun 7 13:10 ./__spark_conf__/capacity-scheduler.xml 111412506 8 -r-x------ 1 yarn hadoop 6935 Jun 7 13:10 ./__spark_conf__/mapred-site.xml 111412531 4 -r-x------ 1 yarn hadoop 2697 Jun 7 13:10 ./__spark_conf__/ssl-server.xml.example 111412528 4 -r-x------ 1 yarn hadoop 535 Jun 7 13:10 ./__spark_conf__/topology_mappings.data 111412508 8 -r-x------ 1 yarn hadoop 6369 Jun 7 13:10 ./__spark_conf__/hadoop-env.sh 111412511 4 -r-x------ 1 yarn hadoop 3348 Jun 7 13:10 ./__spark_conf__/core-site.xml 70255042 4 -rwx------ 1 yarn hadoop 685 Jun 7 13:10 ./default_container_executor_session.sh 111412424 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr 111412425 4 drwx------ 9 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR 111412500 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/R 111412501 976 -r-x------ 1 yarn hadoop 999203 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdb 111412502 24 -r-x------ 1 yarn hadoop 23836 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdx 111412503 4 -r-x------ 1 yarn hadoop 1056 Jun 7 13:10 ./sparkr/SparkR/R/SparkR 111412488 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/Meta 111412490 4 -r-x------ 1 yarn hadoop 188 Jun 7 13:10 ./sparkr/SparkR/Meta/hsearch.rds 111412493 4 -r-x------ 1 yarn hadoop 2467 Jun 7 13:10 ./sparkr/SparkR/Meta/nsInfo.rds 111412492 4 -r-x------ 1 yarn hadoop 38 Jun 7 13:10 ./sparkr/SparkR/Meta/links.rds 111412489 4 -r-x------ 1 yarn hadoop 164 Jun 7 13:10 ./sparkr/SparkR/Meta/Rd.rds 111412491 4 -r-x------ 1 yarn hadoop 1083 Jun 7 13:10 ./sparkr/SparkR/Meta/package.rds 111412454 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests 111412455 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests/testthat 111412482 4 -r-x------ 1 yarn hadoop 1272 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/jarTest.R 111412487 4 -r-x------ 1 yarn hadoop 2388 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Serde.R 111412479 28 -r-x------ 1 yarn hadoop 28197 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_rdd.R 111412485 4 -r-x------ 1 yarn hadoop 1763 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkR.R 111412478 4 -r-x------ 1 yarn hadoop 1332 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 111412480 8 -r-x------ 1 yarn hadoop 8088 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_context.R 111412470 48 -r-x------ 1 yarn hadoop 47315 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_mllib.R 111412475 4 -r-x------ 1 yarn hadoop 3231 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 111412486 12 -r-x------ 1 yarn hadoop 9110 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_utils.R 111412471 8 -r-x------ 1 yarn hadoop 4574 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 111412483 4 -r-x------ 1 yarn hadoop 1084 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Windows.R 111412473 108 -r-x------ 1 yarn hadoop 109097 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 111412467 8 -r-x------ 1 yarn hadoop 8128 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_shuffle.R 111412477 8 -r-x------ 1 yarn hadoop 5331 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_textFile.R 111412481 4 -r-x------ 1 yarn hadoop 1001 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 111412472 4 -r-x------ 1 yarn hadoop 1772 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_broadcast.R 111412456 4 -r-x------ 1 yarn hadoop 1842 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_client.R 111412484 4 -r-x------ 1 yarn hadoop 1971 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_includePackage.R 111412474 4 -r-x------ 1 yarn hadoop 3372 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_take.R 111412476 4 -r-x------ 1 yarn hadoop 3873 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binary_function.R 111412446 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/profile 111412448 4 -r-x------ 1 yarn hadoop 958 Jun 7 13:10 ./sparkr/SparkR/profile/general.R 111412449 4 -r-x------ 1 yarn hadoop 1715 Jun 7 13:10 ./sparkr/SparkR/profile/shell.R 111412429 12 -r-x------ 1 yarn hadoop 10818 Jun 7 13:10 ./sparkr/SparkR/NAMESPACE 111412450 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/html 111412453 4 -r-x------ 1 yarn hadoop 937 Jun 7 13:10 ./sparkr/SparkR/html/00Index.html 111412451 4 -r-x------ 1 yarn hadoop 1319 Jun 7 13:10 ./sparkr/SparkR/html/R.css 111412426 4 -r-x------ 1 yarn hadoop 1431 Jun 7 13:10 ./sparkr/SparkR/DESCRIPTION 111412494 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/help 111412495 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdb 111412499 4 -r-x------ 1 yarn hadoop 60 Jun 7 13:10 ./sparkr/SparkR/help/aliases.rds 111412497 4 -r-x------ 1 yarn hadoop 113 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdx 111412498 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/AnIndex 111412496 4 -r-x------ 1 yarn hadoop 64 Jun 7 13:10 ./sparkr/SparkR/help/paths.rds 111412430 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/worker 111412443 4 -r-x------ 1 yarn hadoop 1975 Jun 7 13:10 ./sparkr/SparkR/worker/daemon.R 111412445 12 -r-x------ 1 yarn hadoop 8950 Jun 7 13:10 ./sparkr/SparkR/worker/worker.R 70255120 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor.sh.crc 70255119 4 -rwx------ 1 yarn hadoop 739 Jun 7 13:10 ./default_container_executor.sh 70255040 4 -rw-r--r-- 1 yarn hadoop 64 Jun 7 13:10 ./.launch_container.sh.crc 70254957 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 13:10 ./.container_tokens.crc 69861606 12 drwxr-xr-x 2 yarn hadoop 12288 Jun 7 12:12 ./__spark_libs__ 69862342 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 69862384 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 69862206 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 69862339 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 69862217 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 69862207 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 69862326 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 69862245 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 69862289 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 69862364 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 69862375 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 69862365 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 69862234 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 69862173 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 69862183 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 69861613 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 69862284 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 69862282 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 69861612 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 69862380 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 69862272 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 69862285 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 69862388 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 69862184 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 69862291 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 69862335 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 69862300 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 69862363 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 69862389 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 69862177 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 69862347 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 69862269 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 69862366 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 69862237 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 69862314 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 69862196 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 69862295 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 69862271 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 69862318 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 69862304 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 69862315 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 69862255 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 69862259 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 69862228 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 69862286 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 69862327 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 69862267 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 69862283 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 69862253 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 69862243 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 69862210 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 69862324 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 69862175 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 69862303 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 69862176 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 69862336 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 69862278 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 69862218 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 69862212 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 69862302 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 69862323 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 69862257 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 69862344 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 69862221 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 69862242 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 69862185 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 69862179 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 69862313 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 69862287 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 69862309 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 69862254 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 69862268 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 69862219 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 69862203 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 69862312 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 69862193 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 69862235 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 69862188 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 69862334 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 69862383 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 69862276 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 69862362 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 69861609 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 69862341 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 69862229 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 69862241 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 69862232 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 69862329 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 69862378 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 69862186 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 69862260 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 69862225 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 69862385 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 69862330 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 69862333 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 69862355 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 69862249 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 69862346 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 69862379 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 69862208 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 69862213 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 69862298 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 69862340 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 69862220 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 69862227 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 69862190 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 69862325 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 69862368 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 69862198 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 69862266 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 69862204 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 69862199 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 69862369 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 69862224 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 69862239 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 69862277 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 69862381 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 69862231 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 69862274 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 69862248 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 69862230 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 69862356 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 69862187 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 69862240 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 69862359 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 69862200 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 69862353 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 69862275 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 69862376 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 69862197 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 69862370 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 69862192 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 69862238 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 69862337 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 69862390 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 69862386 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 69862174 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 69862182 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 69862319 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 69862292 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 69862261 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 69862372 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 69862293 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 69862328 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 69862214 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 69862236 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 69862361 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 69862251 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 69862310 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 69862194 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 69862299 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 69862258 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 69862367 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 69862223 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 69862211 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 69862297 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 69861611 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 69862226 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 69862296 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 69862201 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 69862252 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 69862316 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 69862320 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 69861608 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 69862180 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 69862387 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 69862360 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 69861610 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 69862181 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 69862307 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 69862322 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 69862377 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 69862288 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 69862250 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 69862348 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 69862306 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 69862305 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 69862351 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 69862321 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 69862265 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 69862382 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 69862233 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 69862294 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 69862354 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 69862358 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 69862270 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 69862350 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 69862202 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 69862244 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 69862331 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 69862256 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 69862247 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 69862308 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 69862262 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 69862352 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 69862280 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 69862338 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 69862246 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 69862343 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 69862279 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 69862373 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 69862205 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 69862209 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 69862332 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 69862178 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 69862345 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 69862189 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 69862263 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 69862216 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 69862222 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 69862317 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 69862349 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 69862371 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 69862281 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 69862195 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 69862301 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 69862311 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 69862264 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 69862273 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 69862215 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 69862374 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 69862290 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 69862191 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 69862357 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 70254848 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 ./tmp 70255038 8 -rwx------ 1 yarn hadoop 6680 Jun 7 13:10 ./launch_container.sh 70517401 76 -r-x------ 1 yarn hadoop 74096 Jun 7 13:10 ./py4j-0.10.4-src.zip 70255118 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor_session.sh.crc 70517398 448 -r-x------ 1 yarn hadoop 454814 Jun 7 13:10 ./pyspark.zip broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:6680 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export SPARK_LOG_URL_STDOUT="http://hdfs-hadoop-gs1f.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000006/zeppelin/stdout?start=-4096" export NM_HOST="hdfs-hadoop-gs1f.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export SPARK_LOG_URL_STDERR="http://hdfs-hadoop-gs1f.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000006/zeppelin/stderr?start=-4096" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000006" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/directory.info" exec /bin/bash -c "LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" $JAVA_HOME/bin/java -server -Xmx1024m -Djava.io.tmpdir=$PWD/tmp '-Dspark.history.ui.port=18081' '-Dspark.driver.port=45013' -Dspark.yarn.app.container.log.dir=/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.141.0.9:45013 --executor-id 5 --hostname hdfs-hadoop-gs1f.c.creditvidya-152512.internal --cores 1 --app-id application_1496837077102_0003 --user-class-path file:$PWD/__app__.jar 1>/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/stdout 2>/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000006/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh Container: container_e02_1496837077102_0003_01_000005 on hdfs-hadoop-gs1f.c.creditvidya-152512.internal_45454_1496852894116 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:directory.info Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:36082 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 container_tokens -rwx------ 1 yarn hadoop 685 Jun 7 13:10 default_container_executor_session.sh -rwx------ 1 yarn hadoop 739 Jun 7 13:10 default_container_executor.sh -rwx------ 1 yarn hadoop 6659 Jun 7 13:10 launch_container.sh lrwxrwxrwx 1 yarn hadoop 78 Jun 7 13:10 py4j-0.10.4-src.zip -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 70 Jun 7 13:10 pyspark.zip -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip lrwxrwxrwx 1 yarn hadoop 70 Jun 7 13:10 __spark_conf__ -> /hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 69 Jun 7 13:10 __spark_libs__ -> /hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 62 Jun 7 13:10 sparkr -> /hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 tmp find -L . -maxdepth 5 -ls: 70254823 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 13:10 . 70254844 4 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 ./container_tokens 111412505 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./__spark_conf__ 111412507 4 -r-x------ 1 yarn hadoop 620 Jun 7 13:10 ./__spark_conf__/log4j.properties 111412533 8 -r-x------ 1 yarn hadoop 4113 Jun 7 13:10 ./__spark_conf__/mapred-queues.xml.template 111412532 4 -r-x------ 1 yarn hadoop 945 Jun 7 13:10 ./__spark_conf__/taskcontroller.cfg 111412510 20 -r-x------ 1 yarn hadoop 17718 Jun 7 13:10 ./__spark_conf__/yarn-site.xml 111412516 4 -r-x------ 1 yarn hadoop 1335 Jun 7 13:10 ./__spark_conf__/configuration.xsl 111412514 4 -r-x------ 1 yarn hadoop 758 Jun 7 13:10 ./__spark_conf__/mapred-site.xml.template 111412523 4 -r-x------ 1 yarn hadoop 856 Jun 7 13:10 ./__spark_conf__/mapred-env.sh 111412525 4 -r-x------ 1 yarn hadoop 2316 Jun 7 13:10 ./__spark_conf__/ssl-client.xml.example 111412512 4 -r-x------ 1 yarn hadoop 884 Jun 7 13:10 ./__spark_conf__/ssl-client.xml 111412526 4 -r-x------ 1 yarn hadoop 1308 Jun 7 13:10 ./__spark_conf__/hadoop-policy.xml 111412522 4 -r-x------ 1 yarn hadoop 2490 Jun 7 13:10 ./__spark_conf__/hadoop-metrics.properties 111412520 4 -r-x------ 1 yarn hadoop 1069 Jun 7 13:10 ./__spark_conf__/container-executor.cfg 111412530 4 -r-x------ 1 yarn hadoop 1000 Jun 7 13:10 ./__spark_conf__/ssl-server.xml 111412521 8 -r-x------ 1 yarn hadoop 4221 Jun 7 13:10 ./__spark_conf__/task-log4j.properties 111412529 8 -r-x------ 1 yarn hadoop 4956 Jun 7 13:10 ./__spark_conf__/metrics.properties 111412517 8 -r-x------ 1 yarn hadoop 5305 Jun 7 13:10 ./__spark_conf__/yarn-env.sh 111412519 4 -r-x------ 1 yarn hadoop 1020 Jun 7 13:10 ./__spark_conf__/commons-logging.properties 111412509 4 -r-x------ 1 yarn hadoop 2622 Jun 7 13:10 ./__spark_conf__/hadoop-metrics2.properties 111412515 4 -r-x------ 1 yarn hadoop 2358 Jun 7 13:10 ./__spark_conf__/topology_script.py 111412524 4 -r-x------ 1 yarn hadoop 1602 Jun 7 13:10 ./__spark_conf__/health_check 111412527 4 -r-x------ 1 yarn hadoop 283 Jun 7 13:10 ./__spark_conf__/slaves 111412518 8 -r-x------ 1 yarn hadoop 6787 Jun 7 13:10 ./__spark_conf__/hdfs-site.xml 111412534 4 -r-x------ 1 yarn hadoop 3920 Jun 7 13:10 ./__spark_conf__/__spark_conf__.properties 111412513 4 -r-x------ 1 yarn hadoop 2135 Jun 7 13:10 ./__spark_conf__/capacity-scheduler.xml 111412506 8 -r-x------ 1 yarn hadoop 6935 Jun 7 13:10 ./__spark_conf__/mapred-site.xml 111412531 4 -r-x------ 1 yarn hadoop 2697 Jun 7 13:10 ./__spark_conf__/ssl-server.xml.example 111412528 4 -r-x------ 1 yarn hadoop 535 Jun 7 13:10 ./__spark_conf__/topology_mappings.data 111412508 8 -r-x------ 1 yarn hadoop 6369 Jun 7 13:10 ./__spark_conf__/hadoop-env.sh 111412511 4 -r-x------ 1 yarn hadoop 3348 Jun 7 13:10 ./__spark_conf__/core-site.xml 70255035 4 -rwx------ 1 yarn hadoop 685 Jun 7 13:10 ./default_container_executor_session.sh 111412424 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr 111412425 4 drwx------ 9 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR 111412500 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/R 111412501 976 -r-x------ 1 yarn hadoop 999203 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdb 111412502 24 -r-x------ 1 yarn hadoop 23836 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdx 111412503 4 -r-x------ 1 yarn hadoop 1056 Jun 7 13:10 ./sparkr/SparkR/R/SparkR 111412488 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/Meta 111412490 4 -r-x------ 1 yarn hadoop 188 Jun 7 13:10 ./sparkr/SparkR/Meta/hsearch.rds 111412493 4 -r-x------ 1 yarn hadoop 2467 Jun 7 13:10 ./sparkr/SparkR/Meta/nsInfo.rds 111412492 4 -r-x------ 1 yarn hadoop 38 Jun 7 13:10 ./sparkr/SparkR/Meta/links.rds 111412489 4 -r-x------ 1 yarn hadoop 164 Jun 7 13:10 ./sparkr/SparkR/Meta/Rd.rds 111412491 4 -r-x------ 1 yarn hadoop 1083 Jun 7 13:10 ./sparkr/SparkR/Meta/package.rds 111412454 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests 111412455 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests/testthat 111412482 4 -r-x------ 1 yarn hadoop 1272 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/jarTest.R 111412487 4 -r-x------ 1 yarn hadoop 2388 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Serde.R 111412479 28 -r-x------ 1 yarn hadoop 28197 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_rdd.R 111412485 4 -r-x------ 1 yarn hadoop 1763 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkR.R 111412478 4 -r-x------ 1 yarn hadoop 1332 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 111412480 8 -r-x------ 1 yarn hadoop 8088 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_context.R 111412470 48 -r-x------ 1 yarn hadoop 47315 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_mllib.R 111412475 4 -r-x------ 1 yarn hadoop 3231 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 111412486 12 -r-x------ 1 yarn hadoop 9110 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_utils.R 111412471 8 -r-x------ 1 yarn hadoop 4574 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 111412483 4 -r-x------ 1 yarn hadoop 1084 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Windows.R 111412473 108 -r-x------ 1 yarn hadoop 109097 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 111412467 8 -r-x------ 1 yarn hadoop 8128 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_shuffle.R 111412477 8 -r-x------ 1 yarn hadoop 5331 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_textFile.R 111412481 4 -r-x------ 1 yarn hadoop 1001 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 111412472 4 -r-x------ 1 yarn hadoop 1772 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_broadcast.R 111412456 4 -r-x------ 1 yarn hadoop 1842 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_client.R 111412484 4 -r-x------ 1 yarn hadoop 1971 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_includePackage.R 111412474 4 -r-x------ 1 yarn hadoop 3372 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_take.R 111412476 4 -r-x------ 1 yarn hadoop 3873 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binary_function.R 111412446 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/profile 111412448 4 -r-x------ 1 yarn hadoop 958 Jun 7 13:10 ./sparkr/SparkR/profile/general.R 111412449 4 -r-x------ 1 yarn hadoop 1715 Jun 7 13:10 ./sparkr/SparkR/profile/shell.R 111412429 12 -r-x------ 1 yarn hadoop 10818 Jun 7 13:10 ./sparkr/SparkR/NAMESPACE 111412450 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/html 111412453 4 -r-x------ 1 yarn hadoop 937 Jun 7 13:10 ./sparkr/SparkR/html/00Index.html 111412451 4 -r-x------ 1 yarn hadoop 1319 Jun 7 13:10 ./sparkr/SparkR/html/R.css 111412426 4 -r-x------ 1 yarn hadoop 1431 Jun 7 13:10 ./sparkr/SparkR/DESCRIPTION 111412494 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/help 111412495 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdb 111412499 4 -r-x------ 1 yarn hadoop 60 Jun 7 13:10 ./sparkr/SparkR/help/aliases.rds 111412497 4 -r-x------ 1 yarn hadoop 113 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdx 111412498 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/AnIndex 111412496 4 -r-x------ 1 yarn hadoop 64 Jun 7 13:10 ./sparkr/SparkR/help/paths.rds 111412430 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/worker 111412443 4 -r-x------ 1 yarn hadoop 1975 Jun 7 13:10 ./sparkr/SparkR/worker/daemon.R 111412445 12 -r-x------ 1 yarn hadoop 8950 Jun 7 13:10 ./sparkr/SparkR/worker/worker.R 70255059 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor.sh.crc 70255041 4 -rwx------ 1 yarn hadoop 739 Jun 7 13:10 ./default_container_executor.sh 70254956 4 -rw-r--r-- 1 yarn hadoop 64 Jun 7 13:10 ./.launch_container.sh.crc 70254846 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 13:10 ./.container_tokens.crc 69861606 12 drwxr-xr-x 2 yarn hadoop 12288 Jun 7 12:12 ./__spark_libs__ 69862342 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 69862384 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 69862206 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 69862339 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 69862217 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 69862207 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 69862326 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 69862245 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 69862289 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 69862364 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 69862375 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 69862365 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 69862234 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 69862173 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 69862183 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 69861613 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 69862284 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 69862282 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 69861612 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 69862380 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 69862272 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 69862285 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 69862388 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 69862184 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 69862291 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 69862335 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 69862300 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 69862363 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 69862389 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 69862177 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 69862347 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 69862269 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 69862366 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 69862237 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 69862314 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 69862196 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 69862295 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 69862271 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 69862318 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 69862304 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 69862315 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 69862255 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 69862259 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 69862228 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 69862286 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 69862327 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 69862267 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 69862283 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 69862253 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 69862243 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 69862210 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 69862324 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 69862175 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 69862303 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 69862176 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 69862336 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 69862278 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 69862218 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 69862212 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 69862302 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 69862323 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 69862257 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 69862344 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 69862221 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 69862242 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 69862185 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 69862179 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 69862313 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 69862287 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 69862309 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 69862254 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 69862268 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 69862219 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 69862203 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 69862312 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 69862193 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 69862235 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 69862188 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 69862334 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 69862383 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 69862276 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 69862362 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 69861609 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 69862341 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 69862229 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 69862241 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 69862232 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 69862329 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 69862378 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 69862186 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 69862260 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 69862225 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 69862385 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 69862330 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 69862333 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 69862355 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 69862249 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 69862346 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 69862379 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 69862208 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 69862213 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 69862298 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 69862340 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 69862220 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 69862227 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 69862190 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 69862325 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 69862368 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 69862198 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 69862266 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 69862204 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 69862199 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 69862369 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 69862224 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 69862239 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 69862277 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 69862381 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 69862231 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 69862274 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 69862248 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 69862230 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 69862356 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 69862187 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 69862240 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 69862359 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 69862200 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 69862353 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 69862275 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 69862376 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 69862197 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 69862370 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 69862192 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 69862238 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 69862337 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 69862390 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 69862386 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 69862174 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 69862182 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 69862319 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 69862292 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 69862261 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 69862372 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 69862293 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 69862328 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 69862214 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 69862236 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 69862361 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 69862251 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 69862310 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 69862194 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 69862299 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 69862258 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 69862367 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 69862223 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 69862211 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 69862297 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 69861611 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 69862226 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 69862296 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 69862201 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 69862252 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 69862316 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 69862320 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 69861608 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 69862180 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 69862387 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 69862360 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 69861610 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 69862181 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 69862307 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 69862322 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 69862377 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 69862288 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 69862250 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 69862348 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 69862306 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 69862305 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 69862351 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 69862321 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 69862265 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 69862382 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 69862233 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 69862294 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 69862354 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 69862358 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 69862270 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 69862350 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 69862202 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 69862244 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 69862331 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 69862256 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 69862247 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 69862308 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 69862262 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 69862352 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 69862280 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 69862338 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 69862246 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 69862343 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 69862279 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 69862373 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 69862205 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 69862209 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 69862332 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 69862178 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 69862345 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 69862189 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 69862263 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 69862216 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 69862222 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 69862317 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 69862349 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 69862371 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 69862281 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 69862195 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 69862301 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 69862311 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 69862264 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 69862273 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 69862215 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 69862374 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 69862290 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 69862191 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 69862357 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 70254843 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 ./tmp 70254954 8 -rwx------ 1 yarn hadoop 6659 Jun 7 13:10 ./launch_container.sh 70517401 76 -r-x------ 1 yarn hadoop 74096 Jun 7 13:10 ./py4j-0.10.4-src.zip 70255039 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor_session.sh.crc 70517398 448 -r-x------ 1 yarn hadoop 454814 Jun 7 13:10 ./pyspark.zip broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:6659 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export SPARK_LOG_URL_STDOUT="http://hdfs-hadoop-gs1f.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000005/zeppelin/stdout?start=-4096" export NM_HOST="hdfs-hadoop-gs1f.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export SPARK_LOG_URL_STDERR="http://hdfs-hadoop-gs1f.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000005/zeppelin/stderr?start=-4096" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000005" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/directory.info" exec /bin/bash -c "LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" $JAVA_HOME/bin/java -server -Xmx1024m -Djava.io.tmpdir=$PWD/tmp '-Dspark.history.ui.port=18081' '-Dspark.driver.port=45013' -Dspark.yarn.app.container.log.dir=/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.141.0.9:45013 --executor-id 4 --hostname hdfs-hadoop-gs1f.c.creditvidya-152512.internal --cores 1 --app-id application_1496837077102_0003 --user-class-path file:$PWD/__app__.jar 1>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/stdout 2>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh LogType:stderr Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:7905 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 13:10:49 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 2888@hdfs-hadoop-gs1f 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for TERM 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for HUP 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for INT 17/06/07 13:10:50 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:50 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:50 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 63 ms (0 ms spent in bootstraps) 17/06/07 13:10:50 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:50 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:50 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:50 INFO DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-dc3f1fe3-ffb3-480b-85de-9f1df0a85759 17/06/07 13:10:50 INFO DiskBlockManager: Created local directory at /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-2e763d84-a0d0-4182-9488-096e50ee7355 17/06/07 13:10:50 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.141.0.9:45013 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Successfully registered with driver 17/06/07 13:10:51 INFO Executor: Starting executor ID 4 on host hdfs-hadoop-gs1f.c.creditvidya-152512.internal 17/06/07 13:10:51 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 42192. 17/06/07 13:10:51 INFO NettyBlockTransferService: Server created on hdfs-hadoop-gs1f.c.creditvidya-152512.internal:42192 17/06/07 13:10:51 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/06/07 13:10:51 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(4, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42192, None) 17/06/07 13:10:51 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(4, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42192, None) 17/06/07 13:10:51 INFO BlockManager: external shuffle service port = 7447 17/06/07 13:10:51 INFO BlockManager: Registering executor with local external shuffle service. 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-gs1f.c.creditvidya-152512.internal/10.141.0.6:7447 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO BlockManager: Initialized BlockManager: BlockManagerId(4, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42192, None) 17/06/07 13:10:51 INFO Executor: Using REPL class URI: spark://10.141.0.9:45013/classes 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Got assigned task 1 17/06/07 13:10:51 INFO Executor: Running task 0.0 in stage 0.0 (TID 1) 17/06/07 13:10:51 INFO Executor: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar with timestamp 1496837484445 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO Utils: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-dc860495-5dcb-422d-92b9-65e7e9328fa4/fetchFileTemp6156516291420616701.tmp 17/06/07 13:10:51 INFO Utils: Copying /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-dc860495-5dcb-422d-92b9-65e7e9328fa4/-2200289241496837484445_cache to /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar 17/06/07 13:10:51 INFO Executor: Adding file:/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000005/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to class loader 17/06/07 13:10:51 INFO TorrentBroadcast: Started reading broadcast variable 1 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:44013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 3.3 KB, free 366.3 MB) 17/06/07 13:10:51 INFO TorrentBroadcast: Reading broadcast variable 1 took 113 ms 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 5.4 KB, free 366.3 MB) 17/06/07 13:10:51 INFO HadoopRDD: Input split: hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/tenants/1/installation/2017/03/l0/sms_data.json:0+134217728 17/06/07 13:10:51 INFO TorrentBroadcast: Started reading broadcast variable 0 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 33.0 KB, free 366.3 MB) 17/06/07 13:10:51 INFO TorrentBroadcast: Reading broadcast variable 0 took 11 ms 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 490.9 KB, free 365.8 MB) 17/06/07 13:10:51 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 17/06/07 13:10:51 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 17/06/07 13:10:51 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 17/06/07 13:10:51 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 17/06/07 13:10:51 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 17/06/07 13:10:55 INFO Executor: Finished task 0.0 in stage 0.0 (TID 1). 3717 bytes result sent to driver 17/06/07 13:11:56 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/06/07 13:11:56 INFO DiskBlockManager: Shutdown hook called 17/06/07 13:11:56 INFO ShutdownHookManager: Shutdown hook called 17/06/07 13:11:56 INFO ShutdownHookManager: Deleting directory /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-dc860495-5dcb-422d-92b9-65e7e9328fa4 17/06/07 13:11:56 INFO ShutdownHookManager: Deleting directory /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-9e9a46b2-36a8-4168-a40f-d3031a4ae146 End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout Container: container_e02_1496837077102_0003_01_000007 on hdfs-hadoop-gs1f.c.creditvidya-152512.internal_45454_1496852894116 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:stderr Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:10924 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 3105@hdfs-hadoop-gs1f 17/06/07 13:10:51 INFO SignalUtils: Registered signal handler for TERM 17/06/07 13:10:51 INFO SignalUtils: Registered signal handler for HUP 17/06/07 13:10:51 INFO SignalUtils: Registered signal handler for INT 17/06/07 13:10:51 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:51 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:51 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:51 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:51 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:52 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 69 ms (0 ms spent in bootstraps) 17/06/07 13:10:52 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:52 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:52 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:52 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:52 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:52 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:52 INFO DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-91d91f71-7b79-4c9f-a73d-df90fb1e9338 17/06/07 13:10:52 INFO DiskBlockManager: Created local directory at /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-4322d856-fe17-4c0c-bdbb-48bf2550802a 17/06/07 13:10:52 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 17/06/07 13:10:53 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.141.0.9:45013 17/06/07 13:10:53 INFO CoarseGrainedExecutorBackend: Successfully registered with driver 17/06/07 13:10:53 INFO Executor: Starting executor ID 6 on host hdfs-hadoop-gs1f.c.creditvidya-152512.internal 17/06/07 13:10:53 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 42361. 17/06/07 13:10:53 INFO NettyBlockTransferService: Server created on hdfs-hadoop-gs1f.c.creditvidya-152512.internal:42361 17/06/07 13:10:53 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/06/07 13:10:53 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(6, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42361, None) 17/06/07 13:10:53 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(6, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42361, None) 17/06/07 13:10:53 INFO BlockManager: external shuffle service port = 7447 17/06/07 13:10:53 INFO BlockManager: Registering executor with local external shuffle service. 17/06/07 13:10:53 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-gs1f.c.creditvidya-152512.internal/10.141.0.6:7447 after 9 ms (0 ms spent in bootstraps) 17/06/07 13:10:53 INFO BlockManager: Initialized BlockManager: BlockManagerId(6, hdfs-hadoop-gs1f.c.creditvidya-152512.internal, 42361, None) 17/06/07 13:10:53 INFO Executor: Using REPL class URI: spark://10.141.0.9:45013/classes 17/06/07 13:10:55 INFO CoarseGrainedExecutorBackend: Got assigned task 3 17/06/07 13:10:55 INFO Executor: Running task 3.0 in stage 0.0 (TID 3) 17/06/07 13:10:55 INFO Executor: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar with timestamp 1496837484445 17/06/07 13:10:55 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 3 ms (0 ms spent in bootstraps) 17/06/07 13:10:55 INFO Utils: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-2e36ab44-ea95-4f4f-a53f-101555714365/fetchFileTemp682257188073456663.tmp 17/06/07 13:10:55 INFO Utils: Copying /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-2e36ab44-ea95-4f4f-a53f-101555714365/-2200289241496837484445_cache to /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar 17/06/07 13:10:55 INFO Executor: Adding file:/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to class loader 17/06/07 13:10:55 INFO TorrentBroadcast: Started reading broadcast variable 1 17/06/07 13:10:55 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-gs1f.c.creditvidya-152512.internal/10.141.0.6:42460 after 2 ms (0 ms spent in bootstraps) 17/06/07 13:10:55 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 3.3 KB, free 366.3 MB) 17/06/07 13:10:55 INFO TorrentBroadcast: Reading broadcast variable 1 took 108 ms 17/06/07 13:10:55 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 5.4 KB, free 366.3 MB) 17/06/07 13:10:55 INFO HadoopRDD: Input split: hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/tenants/1/installation/2017/03/l0/sms_data.json:402653184+26159706 17/06/07 13:10:55 INFO TorrentBroadcast: Started reading broadcast variable 0 17/06/07 13:10:55 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 33.0 KB, free 366.3 MB) 17/06/07 13:10:55 INFO TorrentBroadcast: Reading broadcast variable 0 took 13 ms 17/06/07 13:10:55 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 490.9 KB, free 365.8 MB) 17/06/07 13:10:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 17/06/07 13:10:55 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 17/06/07 13:10:55 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 17/06/07 13:10:55 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 17/06/07 13:10:55 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 17/06/07 13:10:57 INFO Executor: Finished task 3.0 in stage 0.0 (TID 3). 3583 bytes result sent to driver 17/06/07 13:11:06 INFO CoarseGrainedExecutorBackend: Got assigned task 4 17/06/07 13:11:06 INFO Executor: Running task 0.0 in stage 1.0 (TID 4) 17/06/07 13:11:06 INFO TorrentBroadcast: Started reading broadcast variable 3 17/06/07 13:11:06 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:44013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:11:06 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 8.5 KB, free 366.3 MB) 17/06/07 13:11:06 INFO TorrentBroadcast: Reading broadcast variable 3 took 19 ms 17/06/07 13:11:06 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 20.5 KB, free 366.3 MB) 17/06/07 13:11:07 INFO CodeGenerator: Code generated in 497.830006 ms 17/06/07 13:11:07 INFO CodeGenerator: Code generated in 77.455529 ms 17/06/07 13:11:07 INFO CodeGenerator: Code generated in 27.27324 ms 17/06/07 13:11:07 INFO FileScanRDD: Reading File path: hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/tenants/1/installation/2017/03/l0/sms_data.json, range: 0-108251798, partition values: [empty row] 17/06/07 13:11:07 INFO CodeGenerator: Code generated in 24.720964 ms 17/06/07 13:11:07 INFO TorrentBroadcast: Started reading broadcast variable 2 17/06/07 13:11:07 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 33.0 KB, free 366.2 MB) 17/06/07 13:11:07 INFO TorrentBroadcast: Reading broadcast variable 2 took 11 ms 17/06/07 13:11:07 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 484.8 KB, free 365.8 MB) 17/06/07 13:11:08 INFO Executor: Finished task 0.0 in stage 1.0 (TID 4). 4671 bytes result sent to driver 17/06/07 13:11:28 INFO CoarseGrainedExecutorBackend: Got assigned task 5 17/06/07 13:11:28 INFO Executor: Running task 0.0 in stage 2.0 (TID 5) 17/06/07 13:11:28 INFO TorrentBroadcast: Started reading broadcast variable 5 17/06/07 13:11:28 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 8.5 KB, free 366.3 MB) 17/06/07 13:11:28 INFO TorrentBroadcast: Reading broadcast variable 5 took 8 ms 17/06/07 13:11:28 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 20.5 KB, free 366.3 MB) 17/06/07 13:11:28 INFO FileScanRDD: Reading File path: hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/tenants/1/installation/2017/03/l0/sms_data.json, range: 0-108251798, partition values: [empty row] 17/06/07 13:11:28 INFO TorrentBroadcast: Started reading broadcast variable 4 17/06/07 13:11:28 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 33.0 KB, free 366.2 MB) 17/06/07 13:11:28 INFO TorrentBroadcast: Reading broadcast variable 4 took 10 ms 17/06/07 13:11:28 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 484.8 KB, free 365.8 MB) 17/06/07 13:11:28 INFO Executor: Finished task 0.0 in stage 2.0 (TID 5). 4671 bytes result sent to driver 17/06/07 13:12:32 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/06/07 13:12:32 INFO DiskBlockManager: Shutdown hook called 17/06/07 13:12:32 INFO ShutdownHookManager: Shutdown hook called 17/06/07 13:12:32 INFO ShutdownHookManager: Deleting directory /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-37a406da-5b78-4cd6-a0b3-cb122690fa5d 17/06/07 13:12:32 INFO ShutdownHookManager: Deleting directory /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-2e36ab44-ea95-4f4f-a53f-101555714365 End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout LogType:directory.info Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:36082 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 container_tokens -rwx------ 1 yarn hadoop 664 Jun 7 13:10 default_container_executor_session.sh -rwx------ 1 yarn hadoop 718 Jun 7 13:10 default_container_executor.sh -rwx------ 1 yarn hadoop 6680 Jun 7 13:10 launch_container.sh lrwxrwxrwx 1 yarn hadoop 78 Jun 7 13:10 py4j-0.10.4-src.zip -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 70 Jun 7 13:10 pyspark.zip -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip lrwxrwxrwx 1 yarn hadoop 70 Jun 7 13:10 __spark_conf__ -> /hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 69 Jun 7 13:10 __spark_libs__ -> /hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 62 Jun 7 13:10 sparkr -> /hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 tmp find -L . -maxdepth 5 -ls: 70254958 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 13:10 . 70255133 4 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 ./container_tokens 111412505 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./__spark_conf__ 111412507 4 -r-x------ 1 yarn hadoop 620 Jun 7 13:10 ./__spark_conf__/log4j.properties 111412533 8 -r-x------ 1 yarn hadoop 4113 Jun 7 13:10 ./__spark_conf__/mapred-queues.xml.template 111412532 4 -r-x------ 1 yarn hadoop 945 Jun 7 13:10 ./__spark_conf__/taskcontroller.cfg 111412510 20 -r-x------ 1 yarn hadoop 17718 Jun 7 13:10 ./__spark_conf__/yarn-site.xml 111412516 4 -r-x------ 1 yarn hadoop 1335 Jun 7 13:10 ./__spark_conf__/configuration.xsl 111412514 4 -r-x------ 1 yarn hadoop 758 Jun 7 13:10 ./__spark_conf__/mapred-site.xml.template 111412523 4 -r-x------ 1 yarn hadoop 856 Jun 7 13:10 ./__spark_conf__/mapred-env.sh 111412525 4 -r-x------ 1 yarn hadoop 2316 Jun 7 13:10 ./__spark_conf__/ssl-client.xml.example 111412512 4 -r-x------ 1 yarn hadoop 884 Jun 7 13:10 ./__spark_conf__/ssl-client.xml 111412526 4 -r-x------ 1 yarn hadoop 1308 Jun 7 13:10 ./__spark_conf__/hadoop-policy.xml 111412522 4 -r-x------ 1 yarn hadoop 2490 Jun 7 13:10 ./__spark_conf__/hadoop-metrics.properties 111412520 4 -r-x------ 1 yarn hadoop 1069 Jun 7 13:10 ./__spark_conf__/container-executor.cfg 111412530 4 -r-x------ 1 yarn hadoop 1000 Jun 7 13:10 ./__spark_conf__/ssl-server.xml 111412521 8 -r-x------ 1 yarn hadoop 4221 Jun 7 13:10 ./__spark_conf__/task-log4j.properties 111412529 8 -r-x------ 1 yarn hadoop 4956 Jun 7 13:10 ./__spark_conf__/metrics.properties 111412517 8 -r-x------ 1 yarn hadoop 5305 Jun 7 13:10 ./__spark_conf__/yarn-env.sh 111412519 4 -r-x------ 1 yarn hadoop 1020 Jun 7 13:10 ./__spark_conf__/commons-logging.properties 111412509 4 -r-x------ 1 yarn hadoop 2622 Jun 7 13:10 ./__spark_conf__/hadoop-metrics2.properties 111412515 4 -r-x------ 1 yarn hadoop 2358 Jun 7 13:10 ./__spark_conf__/topology_script.py 111412524 4 -r-x------ 1 yarn hadoop 1602 Jun 7 13:10 ./__spark_conf__/health_check 111412527 4 -r-x------ 1 yarn hadoop 283 Jun 7 13:10 ./__spark_conf__/slaves 111412518 8 -r-x------ 1 yarn hadoop 6787 Jun 7 13:10 ./__spark_conf__/hdfs-site.xml 111412534 4 -r-x------ 1 yarn hadoop 3920 Jun 7 13:10 ./__spark_conf__/__spark_conf__.properties 111412513 4 -r-x------ 1 yarn hadoop 2135 Jun 7 13:10 ./__spark_conf__/capacity-scheduler.xml 111412506 8 -r-x------ 1 yarn hadoop 6935 Jun 7 13:10 ./__spark_conf__/mapred-site.xml 111412531 4 -r-x------ 1 yarn hadoop 2697 Jun 7 13:10 ./__spark_conf__/ssl-server.xml.example 111412528 4 -r-x------ 1 yarn hadoop 535 Jun 7 13:10 ./__spark_conf__/topology_mappings.data 111412508 8 -r-x------ 1 yarn hadoop 6369 Jun 7 13:10 ./__spark_conf__/hadoop-env.sh 111412511 4 -r-x------ 1 yarn hadoop 3348 Jun 7 13:10 ./__spark_conf__/core-site.xml 70255137 4 -rwx------ 1 yarn hadoop 664 Jun 7 13:10 ./default_container_executor_session.sh 111412424 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr 111412425 4 drwx------ 9 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR 111412500 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/R 111412501 976 -r-x------ 1 yarn hadoop 999203 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdb 111412502 24 -r-x------ 1 yarn hadoop 23836 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdx 111412503 4 -r-x------ 1 yarn hadoop 1056 Jun 7 13:10 ./sparkr/SparkR/R/SparkR 111412488 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/Meta 111412490 4 -r-x------ 1 yarn hadoop 188 Jun 7 13:10 ./sparkr/SparkR/Meta/hsearch.rds 111412493 4 -r-x------ 1 yarn hadoop 2467 Jun 7 13:10 ./sparkr/SparkR/Meta/nsInfo.rds 111412492 4 -r-x------ 1 yarn hadoop 38 Jun 7 13:10 ./sparkr/SparkR/Meta/links.rds 111412489 4 -r-x------ 1 yarn hadoop 164 Jun 7 13:10 ./sparkr/SparkR/Meta/Rd.rds 111412491 4 -r-x------ 1 yarn hadoop 1083 Jun 7 13:10 ./sparkr/SparkR/Meta/package.rds 111412454 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests 111412455 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests/testthat 111412482 4 -r-x------ 1 yarn hadoop 1272 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/jarTest.R 111412487 4 -r-x------ 1 yarn hadoop 2388 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Serde.R 111412479 28 -r-x------ 1 yarn hadoop 28197 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_rdd.R 111412485 4 -r-x------ 1 yarn hadoop 1763 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkR.R 111412478 4 -r-x------ 1 yarn hadoop 1332 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 111412480 8 -r-x------ 1 yarn hadoop 8088 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_context.R 111412470 48 -r-x------ 1 yarn hadoop 47315 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_mllib.R 111412475 4 -r-x------ 1 yarn hadoop 3231 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 111412486 12 -r-x------ 1 yarn hadoop 9110 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_utils.R 111412471 8 -r-x------ 1 yarn hadoop 4574 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 111412483 4 -r-x------ 1 yarn hadoop 1084 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Windows.R 111412473 108 -r-x------ 1 yarn hadoop 109097 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 111412467 8 -r-x------ 1 yarn hadoop 8128 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_shuffle.R 111412477 8 -r-x------ 1 yarn hadoop 5331 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_textFile.R 111412481 4 -r-x------ 1 yarn hadoop 1001 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 111412472 4 -r-x------ 1 yarn hadoop 1772 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_broadcast.R 111412456 4 -r-x------ 1 yarn hadoop 1842 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_client.R 111412484 4 -r-x------ 1 yarn hadoop 1971 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_includePackage.R 111412474 4 -r-x------ 1 yarn hadoop 3372 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_take.R 111412476 4 -r-x------ 1 yarn hadoop 3873 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binary_function.R 111412446 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/profile 111412448 4 -r-x------ 1 yarn hadoop 958 Jun 7 13:10 ./sparkr/SparkR/profile/general.R 111412449 4 -r-x------ 1 yarn hadoop 1715 Jun 7 13:10 ./sparkr/SparkR/profile/shell.R 111412429 12 -r-x------ 1 yarn hadoop 10818 Jun 7 13:10 ./sparkr/SparkR/NAMESPACE 111412450 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/html 111412453 4 -r-x------ 1 yarn hadoop 937 Jun 7 13:10 ./sparkr/SparkR/html/00Index.html 111412451 4 -r-x------ 1 yarn hadoop 1319 Jun 7 13:10 ./sparkr/SparkR/html/R.css 111412426 4 -r-x------ 1 yarn hadoop 1431 Jun 7 13:10 ./sparkr/SparkR/DESCRIPTION 111412494 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/help 111412495 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdb 111412499 4 -r-x------ 1 yarn hadoop 60 Jun 7 13:10 ./sparkr/SparkR/help/aliases.rds 111412497 4 -r-x------ 1 yarn hadoop 113 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdx 111412498 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/AnIndex 111412496 4 -r-x------ 1 yarn hadoop 64 Jun 7 13:10 ./sparkr/SparkR/help/paths.rds 111412430 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/worker 111412443 4 -r-x------ 1 yarn hadoop 1975 Jun 7 13:10 ./sparkr/SparkR/worker/daemon.R 111412445 12 -r-x------ 1 yarn hadoop 8950 Jun 7 13:10 ./sparkr/SparkR/worker/worker.R 70255143 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor.sh.crc 70255139 4 -rwx------ 1 yarn hadoop 718 Jun 7 13:10 ./default_container_executor.sh 70255136 4 -rw-r--r-- 1 yarn hadoop 64 Jun 7 13:10 ./.launch_container.sh.crc 70255134 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 13:10 ./.container_tokens.crc 69861606 12 drwxr-xr-x 2 yarn hadoop 12288 Jun 7 12:12 ./__spark_libs__ 69862342 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 69862384 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 69862206 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 69862339 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 69862217 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 69862207 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 69862326 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 69862245 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 69862289 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 69862364 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 69862375 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 69862365 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 69862234 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 69862173 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 69862183 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 69861613 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 69862284 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 69862282 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 69861612 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 69862380 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 69862272 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 69862285 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 69862388 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 69862184 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 69862291 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 69862335 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 69862300 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 69862363 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 69862389 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 69862177 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 69862347 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 69862269 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 69862366 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 69862237 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 69862314 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 69862196 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 69862295 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 69862271 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 69862318 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 69862304 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 69862315 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 69862255 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 69862259 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 69862228 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 69862286 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 69862327 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 69862267 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 69862283 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 69862253 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 69862243 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 69862210 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 69862324 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 69862175 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 69862303 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 69862176 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 69862336 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 69862278 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 69862218 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 69862212 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 69862302 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 69862323 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 69862257 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 69862344 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 69862221 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 69862242 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 69862185 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 69862179 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 69862313 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 69862287 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 69862309 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 69862254 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 69862268 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 69862219 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 69862203 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 69862312 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 69862193 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 69862235 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 69862188 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 69862334 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 69862383 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 69862276 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 69862362 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 69861609 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 69862341 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 69862229 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 69862241 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 69862232 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 69862329 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 69862378 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 69862186 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 69862260 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 69862225 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 69862385 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 69862330 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 69862333 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 69862355 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 69862249 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 69862346 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 69862379 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 69862208 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 69862213 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 69862298 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 69862340 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 69862220 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 69862227 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 69862190 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 69862325 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 69862368 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 69862198 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 69862266 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 69862204 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 69862199 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 69862369 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 69862224 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 69862239 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 69862277 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 69862381 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 69862231 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 69862274 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 69862248 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 69862230 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 69862356 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 69862187 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 69862240 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 69862359 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 69862200 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 69862353 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 69862275 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 69862376 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 69862197 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 69862370 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 69862192 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 69862238 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 69862337 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 69862390 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 69862386 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 69862174 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 69862182 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 69862319 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 69862292 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 69862261 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 69862372 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 69862293 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 69862328 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 69862214 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 69862236 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 69862361 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 69862251 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 69862310 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 69862194 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 69862299 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 69862258 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 69862367 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 69862223 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 69862211 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 69862297 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 69861611 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 69862226 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 69862296 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 69862201 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 69862252 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 69862316 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 69862320 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 69861608 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 69862180 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 69862387 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 69862360 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 69861610 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 69862181 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 69862307 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 69862322 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 69862377 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 69862288 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 69862250 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 69862348 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 69862306 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 69862305 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 69862351 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 69862321 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 69862265 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 69862382 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 69862233 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 69862294 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 69862354 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 69862358 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 69862270 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 69862350 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 69862202 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 69862244 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 69862331 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 69862256 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 69862247 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 69862308 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 69862262 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 69862352 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 69862280 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 69862338 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 69862246 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 69862343 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 69862279 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 69862373 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 69862205 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 69862209 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 69862332 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 69862178 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 69862345 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 69862189 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 69862263 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 69862216 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 69862222 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 69862317 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 69862349 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 69862371 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 69862281 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 69862195 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 69862301 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 69862311 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 69862264 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 69862273 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 69862215 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 69862374 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 69862290 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 69862191 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 69862357 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 70255132 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 ./tmp 70255135 8 -rwx------ 1 yarn hadoop 6680 Jun 7 13:10 ./launch_container.sh 70517401 76 -r-x------ 1 yarn hadoop 74096 Jun 7 13:10 ./py4j-0.10.4-src.zip 70255138 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor_session.sh.crc 70517398 448 -r-x------ 1 yarn hadoop 454814 Jun 7 13:10 ./pyspark.zip broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:6680 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export SPARK_LOG_URL_STDOUT="http://hdfs-hadoop-gs1f.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000007/zeppelin/stdout?start=-4096" export NM_HOST="hdfs-hadoop-gs1f.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export SPARK_LOG_URL_STDERR="http://hdfs-hadoop-gs1f.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000007/zeppelin/stderr?start=-4096" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000007" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/213/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/215/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/216/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/214/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/directory.info" exec /bin/bash -c "LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" $JAVA_HOME/bin/java -server -Xmx1024m -Djava.io.tmpdir=$PWD/tmp '-Dspark.history.ui.port=18081' '-Dspark.driver.port=45013' -Dspark.yarn.app.container.log.dir=/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.141.0.9:45013 --executor-id 6 --hostname hdfs-hadoop-gs1f.c.creditvidya-152512.internal --cores 1 --app-id application_1496837077102_0003 --user-class-path file:$PWD/__app__.jar 1>/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/stdout 2>/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000007/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh Container: container_e02_1496837077102_0003_01_000004 on hdfs-hadoop-mr2w.c.creditvidya-152512.internal_45454_1496852893999 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:directory.info Log Upload Time:Wed Jun 07 16:28:13 +0000 2017 LogLength:35994 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 container_tokens -rwx------ 1 yarn hadoop 678 Jun 7 13:10 default_container_executor_session.sh -rwx------ 1 yarn hadoop 732 Jun 7 13:10 default_container_executor.sh -rwx------ 1 yarn hadoop 6638 Jun 7 13:10 launch_container.sh lrwxrwxrwx 1 yarn hadoop 71 Jun 7 13:10 py4j-0.10.4-src.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/195/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 63 Jun 7 13:10 pyspark.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/193/pyspark.zip lrwxrwxrwx 1 yarn hadoop 77 Jun 7 13:10 __spark_conf__ -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/194/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 62 Jun 7 13:10 __spark_libs__ -> /hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 69 Jun 7 13:10 sparkr -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/192/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 tmp find -L . -maxdepth 5 -ls: 73138667 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 13:10 . 73138723 448 -r-x------ 1 yarn hadoop 454814 Jun 7 13:10 ./pyspark.zip 73138937 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor_session.sh.crc 73138936 4 -rwx------ 1 yarn hadoop 678 Jun 7 13:10 ./default_container_executor_session.sh 73138932 4 -rw-r--r-- 1 yarn hadoop 84 Jun 7 13:10 ./container_tokens 73138895 76 -r-x------ 1 yarn hadoop 74096 Jun 7 13:10 ./py4j-0.10.4-src.zip 73138935 4 -rw-r--r-- 1 yarn hadoop 60 Jun 7 13:10 ./.launch_container.sh.crc 70124647 16 drwxr-xr-x 2 yarn hadoop 16384 Jun 7 12:12 ./__spark_libs__ 70124797 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 70124648 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 70124769 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 70124789 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 70124828 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 70124813 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 70124783 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 70124668 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 70124822 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 70124773 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 70124693 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 70124784 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 70124686 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 70124657 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 70124702 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 70124763 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 70124788 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 70124770 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 70124721 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 70124718 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 70124772 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 70124656 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 70124791 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 70124771 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 70124765 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 70124867 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 70124863 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 70124725 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 70124653 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 70124827 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 70124848 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 70124681 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 70124743 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 70124697 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 70124674 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 70124841 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 70124669 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 70124792 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 70124655 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 70124696 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 70124798 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 70124868 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 70124733 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 70124703 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 70124744 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 70124870 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 70124713 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 70124660 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 70124864 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 70124652 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 70124786 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 70124807 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 70124847 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 70124815 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 70124851 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 70124747 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 70124672 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 70124838 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 70124746 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 70124734 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 70124730 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 70124805 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 70124722 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 70124741 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 70124811 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 70124845 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 70124869 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 70124740 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 70124853 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 70124649 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 70124826 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 70124820 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 70124654 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 70124862 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 70124727 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 70124748 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 70124719 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 70124685 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 70124812 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 70124850 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 70124695 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 70124854 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 70124728 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 70124663 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 70124687 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 70124650 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 70124785 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 70124842 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 70124698 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 70124758 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 70124735 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 70124700 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 70124682 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 70124678 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 70124677 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 70124673 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 70124766 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 70124737 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 70124796 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 70124723 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 70124793 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 70124680 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 70124726 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 70124780 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 70124833 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 70124711 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 70124717 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 70124810 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 70124701 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 70124666 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 70124753 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 70124790 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 70124860 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 70124739 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 70124667 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 70124691 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 70124754 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 70124736 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 70124755 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 70124846 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 70124800 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 70124759 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 70124824 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 70124694 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 70124724 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 70124745 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 70124671 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 70124751 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 70124859 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 70124808 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 70124834 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 70124823 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 70124795 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 70124706 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 70124729 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 70124803 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 70124844 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 70124689 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 70124715 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 70124816 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 70124819 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 70124825 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 70124774 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 70124757 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 70124849 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 70124750 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 70124764 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 70124801 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 70124856 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 70124865 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 70124707 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 70124714 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 70124837 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 70124775 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 70124708 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 70124802 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 70124768 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 70124756 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 70124670 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 70124651 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 70124818 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 70124664 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 70124804 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 70124852 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 70124794 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 70124840 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 70124661 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 70124831 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 70124675 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 70124767 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 70124799 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 70124679 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 70124665 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 70124836 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 70124861 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 70124712 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 70124720 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 70124658 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 70124787 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 70124662 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 70124829 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 70124688 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 70124871 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 70124690 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 70124777 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 70124731 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 70124817 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 70124821 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 70124705 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 70124760 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 70124659 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 70124699 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 70124710 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 70124857 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 70124843 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 70124855 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 70124762 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 70124809 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 70124752 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 70124830 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 70124839 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 70124866 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 70124709 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 70124676 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 70124835 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 70124776 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 70124716 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 70124684 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 70124742 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 70124781 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 70124738 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 70124692 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 70124858 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 70124814 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 70124683 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 70124782 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 70124779 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 70124832 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 70124761 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 70124778 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 70124732 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 70124749 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 70124806 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 70124704 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 73138939 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 13:10 ./.default_container_executor.sh.crc 73138896 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./__spark_conf__ 73138907 4 -r-x------ 1 yarn hadoop 2135 Jun 7 13:10 ./__spark_conf__/capacity-scheduler.xml 73138914 4 -r-x------ 1 yarn hadoop 1069 Jun 7 13:10 ./__spark_conf__/container-executor.cfg 73138915 8 -r-x------ 1 yarn hadoop 4221 Jun 7 13:10 ./__spark_conf__/task-log4j.properties 73138923 8 -r-x------ 1 yarn hadoop 4956 Jun 7 13:10 ./__spark_conf__/metrics.properties 73138925 4 -r-x------ 1 yarn hadoop 2697 Jun 7 13:10 ./__spark_conf__/ssl-server.xml.example 73138926 4 -r-x------ 1 yarn hadoop 945 Jun 7 13:10 ./__spark_conf__/taskcontroller.cfg 73138919 4 -r-x------ 1 yarn hadoop 2316 Jun 7 13:10 ./__spark_conf__/ssl-client.xml.example 73138901 20 -r-x------ 1 yarn hadoop 17718 Jun 7 13:10 ./__spark_conf__/yarn-site.xml 73138916 4 -r-x------ 1 yarn hadoop 2490 Jun 7 13:10 ./__spark_conf__/hadoop-metrics.properties 73138927 8 -r-x------ 1 yarn hadoop 4113 Jun 7 13:10 ./__spark_conf__/mapred-queues.xml.template 73138920 4 -r-x------ 1 yarn hadoop 1308 Jun 7 13:10 ./__spark_conf__/hadoop-policy.xml 73138898 4 -r-x------ 1 yarn hadoop 620 Jun 7 13:10 ./__spark_conf__/log4j.properties 73138910 4 -r-x------ 1 yarn hadoop 1335 Jun 7 13:10 ./__spark_conf__/configuration.xsl 73138913 4 -r-x------ 1 yarn hadoop 1020 Jun 7 13:10 ./__spark_conf__/commons-logging.properties 73138902 4 -r-x------ 1 yarn hadoop 3348 Jun 7 13:10 ./__spark_conf__/core-site.xml 73138928 4 -r-x------ 1 yarn hadoop 3920 Jun 7 13:10 ./__spark_conf__/__spark_conf__.properties 73138911 8 -r-x------ 1 yarn hadoop 5305 Jun 7 13:10 ./__spark_conf__/yarn-env.sh 73138897 8 -r-x------ 1 yarn hadoop 6935 Jun 7 13:10 ./__spark_conf__/mapred-site.xml 73138918 4 -r-x------ 1 yarn hadoop 1602 Jun 7 13:10 ./__spark_conf__/health_check 73138921 4 -r-x------ 1 yarn hadoop 283 Jun 7 13:10 ./__spark_conf__/slaves 73138899 8 -r-x------ 1 yarn hadoop 6369 Jun 7 13:10 ./__spark_conf__/hadoop-env.sh 73138922 4 -r-x------ 1 yarn hadoop 535 Jun 7 13:10 ./__spark_conf__/topology_mappings.data 73138909 4 -r-x------ 1 yarn hadoop 2358 Jun 7 13:10 ./__spark_conf__/topology_script.py 73138900 4 -r-x------ 1 yarn hadoop 2622 Jun 7 13:10 ./__spark_conf__/hadoop-metrics2.properties 73138906 4 -r-x------ 1 yarn hadoop 884 Jun 7 13:10 ./__spark_conf__/ssl-client.xml 73138924 4 -r-x------ 1 yarn hadoop 1000 Jun 7 13:10 ./__spark_conf__/ssl-server.xml 73138908 4 -r-x------ 1 yarn hadoop 758 Jun 7 13:10 ./__spark_conf__/mapred-site.xml.template 73138917 4 -r-x------ 1 yarn hadoop 856 Jun 7 13:10 ./__spark_conf__/mapred-env.sh 73138912 8 -r-x------ 1 yarn hadoop 6787 Jun 7 13:10 ./__spark_conf__/hdfs-site.xml 73138746 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr 73138747 4 drwx------ 9 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR 73138752 12 -r-x------ 1 yarn hadoop 10818 Jun 7 13:10 ./sparkr/SparkR/NAMESPACE 73138853 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/Meta 73138860 4 -r-x------ 1 yarn hadoop 164 Jun 7 13:10 ./sparkr/SparkR/Meta/Rd.rds 73138868 4 -r-x------ 1 yarn hadoop 38 Jun 7 13:10 ./sparkr/SparkR/Meta/links.rds 73138861 4 -r-x------ 1 yarn hadoop 188 Jun 7 13:10 ./sparkr/SparkR/Meta/hsearch.rds 73138867 4 -r-x------ 1 yarn hadoop 1083 Jun 7 13:10 ./sparkr/SparkR/Meta/package.rds 73138869 4 -r-x------ 1 yarn hadoop 2467 Jun 7 13:10 ./sparkr/SparkR/Meta/nsInfo.rds 73138801 4 drwx------ 3 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests 73138802 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/tests/testthat 73138806 8 -r-x------ 1 yarn hadoop 4574 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 73138831 4 -r-x------ 1 yarn hadoop 1272 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/jarTest.R 73138817 4 -r-x------ 1 yarn hadoop 1332 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 73138829 8 -r-x------ 1 yarn hadoop 8088 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_context.R 73138850 4 -r-x------ 1 yarn hadoop 1763 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkR.R 73138811 4 -r-x------ 1 yarn hadoop 3231 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 73138808 4 -r-x------ 1 yarn hadoop 1772 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_broadcast.R 73138804 8 -r-x------ 1 yarn hadoop 8128 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_shuffle.R 73138810 4 -r-x------ 1 yarn hadoop 3372 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_take.R 73138805 48 -r-x------ 1 yarn hadoop 47315 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_mllib.R 73138816 8 -r-x------ 1 yarn hadoop 5331 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_textFile.R 73138852 4 -r-x------ 1 yarn hadoop 2388 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Serde.R 73138828 28 -r-x------ 1 yarn hadoop 28197 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_rdd.R 73138833 4 -r-x------ 1 yarn hadoop 1084 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_Windows.R 73138803 4 -r-x------ 1 yarn hadoop 1842 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_client.R 73138851 12 -r-x------ 1 yarn hadoop 9110 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_utils.R 73138830 4 -r-x------ 1 yarn hadoop 1001 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 73138842 4 -r-x------ 1 yarn hadoop 1971 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_includePackage.R 73138809 108 -r-x------ 1 yarn hadoop 109097 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 73138815 4 -r-x------ 1 yarn hadoop 3873 Jun 7 13:10 ./sparkr/SparkR/tests/testthat/test_binary_function.R 73138765 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/html 73138800 4 -r-x------ 1 yarn hadoop 937 Jun 7 13:10 ./sparkr/SparkR/html/00Index.html 73138799 4 -r-x------ 1 yarn hadoop 1319 Jun 7 13:10 ./sparkr/SparkR/html/R.css 73138883 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/R 73138888 24 -r-x------ 1 yarn hadoop 23836 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdx 73138884 976 -r-x------ 1 yarn hadoop 999203 Jun 7 13:10 ./sparkr/SparkR/R/SparkR.rdb 73138889 4 -r-x------ 1 yarn hadoop 1056 Jun 7 13:10 ./sparkr/SparkR/R/SparkR 73138751 4 -r-x------ 1 yarn hadoop 1431 Jun 7 13:10 ./sparkr/SparkR/DESCRIPTION 73138760 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/profile 73138761 4 -r-x------ 1 yarn hadoop 958 Jun 7 13:10 ./sparkr/SparkR/profile/general.R 73138762 4 -r-x------ 1 yarn hadoop 1715 Jun 7 13:10 ./sparkr/SparkR/profile/shell.R 73138872 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/help 73138881 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/AnIndex 73138880 4 -r-x------ 1 yarn hadoop 113 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdx 73138878 0 -r-x------ 1 yarn hadoop 0 Jun 7 13:10 ./sparkr/SparkR/help/SparkR.rdb 73138882 4 -r-x------ 1 yarn hadoop 60 Jun 7 13:10 ./sparkr/SparkR/help/aliases.rds 73138879 4 -r-x------ 1 yarn hadoop 64 Jun 7 13:10 ./sparkr/SparkR/help/paths.rds 73138753 4 drwx------ 2 yarn hadoop 4096 Jun 7 13:10 ./sparkr/SparkR/worker 73138755 4 -r-x------ 1 yarn hadoop 1975 Jun 7 13:10 ./sparkr/SparkR/worker/daemon.R 73138756 12 -r-x------ 1 yarn hadoop 8950 Jun 7 13:10 ./sparkr/SparkR/worker/worker.R 73138934 8 -rwx------ 1 yarn hadoop 6638 Jun 7 13:10 ./launch_container.sh 73138933 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 13:10 ./.container_tokens.crc 73138938 4 -rwx------ 1 yarn hadoop 732 Jun 7 13:10 ./default_container_executor.sh 73138931 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 13:10 ./tmp broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:13 +0000 2017 LogLength:6638 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export SPARK_LOG_URL_STDOUT="http://hdfs-hadoop-mr2w.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000004/zeppelin/stdout?start=-4096" export NM_HOST="hdfs-hadoop-mr2w.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export SPARK_LOG_URL_STDERR="http://hdfs-hadoop-mr2w.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000004/zeppelin/stderr?start=-4096" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000004" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/193/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/194/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/192/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/195/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/directory.info" exec /bin/bash -c "LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" $JAVA_HOME/bin/java -server -Xmx1024m -Djava.io.tmpdir=$PWD/tmp '-Dspark.history.ui.port=18081' '-Dspark.driver.port=45013' -Dspark.yarn.app.container.log.dir=/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.141.0.9:45013 --executor-id 3 --hostname hdfs-hadoop-mr2w.c.creditvidya-152512.internal --cores 1 --app-id application_1496837077102_0003 --user-class-path file:$PWD/__app__.jar 1>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/stdout 2>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh LogType:stderr Log Upload Time:Wed Jun 07 16:28:13 +0000 2017 LogLength:8076 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/yarn/local/filecache/11/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 13:10:49 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 4558@hdfs-hadoop-mr2w 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for TERM 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for HUP 17/06/07 13:10:49 INFO SignalUtils: Registered signal handler for INT 17/06/07 13:10:50 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:50 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:50 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 77 ms (0 ms spent in bootstraps) 17/06/07 13:10:50 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 13:10:50 INFO SecurityManager: Changing view acls groups to: 17/06/07 13:10:50 INFO SecurityManager: Changing modify acls groups to: 17/06/07 13:10:50 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 13:10:50 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 2 ms (0 ms spent in bootstraps) 17/06/07 13:10:50 INFO DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-820fbac1-17ae-4781-a35e-bbb20fcbea2d 17/06/07 13:10:50 INFO DiskBlockManager: Created local directory at /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-6ab31933-6205-46de-81d2-e332f0459c10 17/06/07 13:10:50 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.141.0.9:45013 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Successfully registered with driver 17/06/07 13:10:51 INFO Executor: Starting executor ID 3 on host hdfs-hadoop-mr2w.c.creditvidya-152512.internal 17/06/07 13:10:51 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 44584. 17/06/07 13:10:51 INFO NettyBlockTransferService: Server created on hdfs-hadoop-mr2w.c.creditvidya-152512.internal:44584 17/06/07 13:10:51 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/06/07 13:10:51 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(3, hdfs-hadoop-mr2w.c.creditvidya-152512.internal, 44584, None) 17/06/07 13:10:51 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(3, hdfs-hadoop-mr2w.c.creditvidya-152512.internal, 44584, None) 17/06/07 13:10:51 INFO BlockManager: external shuffle service port = 7447 17/06/07 13:10:51 INFO BlockManager: Registering executor with local external shuffle service. 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-mr2w.c.creditvidya-152512.internal/10.141.0.8:7447 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO BlockManager: Initialized BlockManager: BlockManagerId(3, hdfs-hadoop-mr2w.c.creditvidya-152512.internal, 44584, None) 17/06/07 13:10:51 INFO Executor: Using REPL class URI: spark://10.141.0.9:45013/classes 17/06/07 13:10:51 INFO CoarseGrainedExecutorBackend: Got assigned task 0 17/06/07 13:10:51 INFO Executor: Running task 1.0 in stage 0.0 (TID 0) 17/06/07 13:10:51 INFO Executor: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar with timestamp 1496837484445 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO Utils: Fetching spark://10.141.0.9:45013/jars/zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-ec26a189-5353-48cb-8a66-f8afc2cce303/fetchFileTemp6973848409089263542.tmp 17/06/07 13:10:51 INFO Utils: Copying /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-ec26a189-5353-48cb-8a66-f8afc2cce303/-2200289241496837484445_cache to /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar 17/06/07 13:10:51 INFO Executor: Adding file:/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000004/./zeppelin-spark_2.10-0.7.0.2.6.0.3-8.jar to class loader 17/06/07 13:10:51 INFO TorrentBroadcast: Started reading broadcast variable 1 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:44013 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 3.3 KB, free 366.3 MB) 17/06/07 13:10:51 INFO TorrentBroadcast: Reading broadcast variable 1 took 142 ms 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 5.4 KB, free 366.3 MB) 17/06/07 13:10:51 INFO HadoopRDD: Input split: hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/tenants/1/installation/2017/03/l0/sms_data.json:134217728+134217728 17/06/07 13:10:51 INFO TorrentBroadcast: Started reading broadcast variable 0 17/06/07 13:10:51 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-gs1f.c.creditvidya-152512.internal/10.141.0.6:42192 after 1 ms (0 ms spent in bootstraps) 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 33.0 KB, free 366.3 MB) 17/06/07 13:10:51 INFO TorrentBroadcast: Reading broadcast variable 0 took 19 ms 17/06/07 13:10:51 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 490.9 KB, free 365.8 MB) 17/06/07 13:10:52 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 17/06/07 13:10:52 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 17/06/07 13:10:52 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 17/06/07 13:10:52 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 17/06/07 13:10:52 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 17/06/07 13:10:55 INFO Executor: Finished task 1.0 in stage 0.0 (TID 0). 3717 bytes result sent to driver 17/06/07 13:11:56 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/06/07 13:11:56 INFO DiskBlockManager: Shutdown hook called 17/06/07 13:11:56 INFO ShutdownHookManager: Shutdown hook called 17/06/07 13:11:56 INFO ShutdownHookManager: Deleting directory /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-ec26a189-5353-48cb-8a66-f8afc2cce303 17/06/07 13:11:56 INFO ShutdownHookManager: Deleting directory /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/spark-dc97bb6b-ef6f-4040-a570-2b69bf9569d0 End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:13 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout Container: container_e02_1496837077102_0003_01_000003 on hdfs-hadoop-p0z7.c.creditvidya-152512.internal_45454_1496852894001 LogAggregationType: AGGREGATED =========================================================================================================================== LogType:directory.info Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:35999 Log Contents: ls -l: total 44 -rw-r--r-- 1 yarn hadoop 84 Jun 7 12:11 container_tokens -rwx------ 1 yarn hadoop 685 Jun 7 12:11 default_container_executor_session.sh -rwx------ 1 yarn hadoop 739 Jun 7 12:11 default_container_executor.sh -rwx------ 1 yarn hadoop 6655 Jun 7 12:11 launch_container.sh lrwxrwxrwx 1 yarn hadoop 70 Jun 7 12:11 py4j-0.10.4-src.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/37/py4j-0.10.4-src.zip lrwxrwxrwx 1 yarn hadoop 62 Jun 7 12:11 pyspark.zip -> /hadoop/yarn/local/usercache/zeppelin/filecache/35/pyspark.zip lrwxrwxrwx 1 yarn hadoop 76 Jun 7 12:11 __spark_conf__ -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/36/__spark_conf__.zip lrwxrwxrwx 1 yarn hadoop 69 Jun 7 12:11 __spark_libs__ -> /hadoop/hadoop/yarn/local/filecache/10/spark2-hdp-yarn-archive.tar.gz lrwxrwxrwx 1 yarn hadoop 68 Jun 7 12:11 sparkr -> /hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/34/sparkr.zip drwx--x--- 2 yarn hadoop 4096 Jun 7 12:11 tmp find -L . -maxdepth 5 -ls: 46400664 4 drwx--x--- 3 yarn hadoop 4096 Jun 7 12:11 . 46400674 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 12:11 ./.default_container_executor.sh.crc 46400670 4 -rw-r--r-- 1 yarn hadoop 60 Jun 7 12:11 ./.launch_container.sh.crc 46400667 4 -rw-r--r-- 1 yarn hadoop 84 Jun 7 12:11 ./container_tokens 46400672 4 -rw-r--r-- 1 yarn hadoop 16 Jun 7 12:11 ./.default_container_executor_session.sh.crc 46400669 8 -rwx------ 1 yarn hadoop 6655 Jun 7 12:11 ./launch_container.sh 46400673 4 -rwx------ 1 yarn hadoop 739 Jun 7 12:11 ./default_container_executor.sh 46400666 4 drwx--x--- 2 yarn hadoop 4096 Jun 7 12:11 ./tmp 115606207 448 -r-x------ 1 yarn hadoop 454814 Jun 7 12:11 ./pyspark.zip 46400671 4 -rwx------ 1 yarn hadoop 685 Jun 7 12:11 ./default_container_executor_session.sh 46400633 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./__spark_conf__ 46400636 8 -r-x------ 1 yarn hadoop 6369 Jun 7 12:11 ./__spark_conf__/hadoop-env.sh 46400655 4 -r-x------ 1 yarn hadoop 283 Jun 7 12:11 ./__spark_conf__/slaves 46400639 4 -r-x------ 1 yarn hadoop 3348 Jun 7 12:11 ./__spark_conf__/core-site.xml 46400657 8 -r-x------ 1 yarn hadoop 4956 Jun 7 12:11 ./__spark_conf__/metrics.properties 46400662 4 -r-x------ 1 yarn hadoop 3920 Jun 7 12:11 ./__spark_conf__/__spark_conf__.properties 46400660 4 -r-x------ 1 yarn hadoop 945 Jun 7 12:11 ./__spark_conf__/taskcontroller.cfg 46400661 8 -r-x------ 1 yarn hadoop 4113 Jun 7 12:11 ./__spark_conf__/mapred-queues.xml.template 46400653 4 -r-x------ 1 yarn hadoop 2316 Jun 7 12:11 ./__spark_conf__/ssl-client.xml.example 46400642 4 -r-x------ 1 yarn hadoop 758 Jun 7 12:11 ./__spark_conf__/mapred-site.xml.template 46400641 4 -r-x------ 1 yarn hadoop 2135 Jun 7 12:11 ./__spark_conf__/capacity-scheduler.xml 46400650 4 -r-x------ 1 yarn hadoop 2490 Jun 7 12:11 ./__spark_conf__/hadoop-metrics.properties 46400652 4 -r-x------ 1 yarn hadoop 1602 Jun 7 12:11 ./__spark_conf__/health_check 46400659 4 -r-x------ 1 yarn hadoop 2697 Jun 7 12:11 ./__spark_conf__/ssl-server.xml.example 46400656 4 -r-x------ 1 yarn hadoop 535 Jun 7 12:11 ./__spark_conf__/topology_mappings.data 46400654 4 -r-x------ 1 yarn hadoop 1308 Jun 7 12:11 ./__spark_conf__/hadoop-policy.xml 46400649 8 -r-x------ 1 yarn hadoop 4221 Jun 7 12:11 ./__spark_conf__/task-log4j.properties 46400645 8 -r-x------ 1 yarn hadoop 5305 Jun 7 12:11 ./__spark_conf__/yarn-env.sh 46400646 8 -r-x------ 1 yarn hadoop 6787 Jun 7 12:11 ./__spark_conf__/hdfs-site.xml 46400638 20 -r-x------ 1 yarn hadoop 17718 Jun 7 12:11 ./__spark_conf__/yarn-site.xml 46400635 4 -r-x------ 1 yarn hadoop 620 Jun 7 12:11 ./__spark_conf__/log4j.properties 46400634 8 -r-x------ 1 yarn hadoop 6935 Jun 7 12:11 ./__spark_conf__/mapred-site.xml 46400647 4 -r-x------ 1 yarn hadoop 1020 Jun 7 12:11 ./__spark_conf__/commons-logging.properties 46400643 4 -r-x------ 1 yarn hadoop 2358 Jun 7 12:11 ./__spark_conf__/topology_script.py 46400651 4 -r-x------ 1 yarn hadoop 856 Jun 7 12:11 ./__spark_conf__/mapred-env.sh 46400648 4 -r-x------ 1 yarn hadoop 1069 Jun 7 12:11 ./__spark_conf__/container-executor.cfg 46400640 4 -r-x------ 1 yarn hadoop 884 Jun 7 12:11 ./__spark_conf__/ssl-client.xml 46400637 4 -r-x------ 1 yarn hadoop 2622 Jun 7 12:11 ./__spark_conf__/hadoop-metrics2.properties 46400644 4 -r-x------ 1 yarn hadoop 1335 Jun 7 12:11 ./__spark_conf__/configuration.xsl 46400658 4 -r-x------ 1 yarn hadoop 1000 Jun 7 12:11 ./__spark_conf__/ssl-server.xml 46400558 4 drwx------ 3 yarn hadoop 4096 Jun 7 12:11 ./sparkr 46400559 4 drwx------ 9 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR 46400616 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/Meta 46400617 4 -r-x------ 1 yarn hadoop 164 Jun 7 12:11 ./sparkr/SparkR/Meta/Rd.rds 46400620 4 -r-x------ 1 yarn hadoop 38 Jun 7 12:11 ./sparkr/SparkR/Meta/links.rds 46400621 4 -r-x------ 1 yarn hadoop 2467 Jun 7 12:11 ./sparkr/SparkR/Meta/nsInfo.rds 46400618 4 -r-x------ 1 yarn hadoop 188 Jun 7 12:11 ./sparkr/SparkR/Meta/hsearch.rds 46400619 4 -r-x------ 1 yarn hadoop 1083 Jun 7 12:11 ./sparkr/SparkR/Meta/package.rds 46400594 4 drwx------ 3 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/tests 46400595 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/tests/testthat 46400599 8 -r-x------ 1 yarn hadoop 4574 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_parallelize_collect.R 46400602 4 -r-x------ 1 yarn hadoop 3372 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_take.R 46400606 4 -r-x------ 1 yarn hadoop 1332 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_jvm_api.R 46400604 4 -r-x------ 1 yarn hadoop 3873 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_binary_function.R 46400612 4 -r-x------ 1 yarn hadoop 1971 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_includePackage.R 46400596 4 -r-x------ 1 yarn hadoop 1842 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_client.R 46400608 8 -r-x------ 1 yarn hadoop 8088 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_context.R 46400609 4 -r-x------ 1 yarn hadoop 1001 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/packageInAJarTest.R 46400610 4 -r-x------ 1 yarn hadoop 1272 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/jarTest.R 46400603 4 -r-x------ 1 yarn hadoop 3231 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_binaryFile.R 46400613 4 -r-x------ 1 yarn hadoop 1763 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_sparkR.R 46400605 8 -r-x------ 1 yarn hadoop 5331 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_textFile.R 46400600 4 -r-x------ 1 yarn hadoop 1772 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_broadcast.R 46400598 48 -r-x------ 1 yarn hadoop 47315 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_mllib.R 46400597 8 -r-x------ 1 yarn hadoop 8128 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_shuffle.R 46400614 12 -r-x------ 1 yarn hadoop 9110 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_utils.R 46400611 4 -r-x------ 1 yarn hadoop 1084 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_Windows.R 46400601 108 -r-x------ 1 yarn hadoop 109097 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_sparkSQL.R 46400615 4 -r-x------ 1 yarn hadoop 2388 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_Serde.R 46400607 28 -r-x------ 1 yarn hadoop 28197 Jun 7 12:11 ./sparkr/SparkR/tests/testthat/test_rdd.R 46400561 12 -r-x------ 1 yarn hadoop 10818 Jun 7 12:11 ./sparkr/SparkR/NAMESPACE 46400562 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/worker 46400564 12 -r-x------ 1 yarn hadoop 8950 Jun 7 12:11 ./sparkr/SparkR/worker/worker.R 46400563 4 -r-x------ 1 yarn hadoop 1975 Jun 7 12:11 ./sparkr/SparkR/worker/daemon.R 46400565 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/profile 46400589 4 -r-x------ 1 yarn hadoop 958 Jun 7 12:11 ./sparkr/SparkR/profile/general.R 46400590 4 -r-x------ 1 yarn hadoop 1715 Jun 7 12:11 ./sparkr/SparkR/profile/shell.R 46400591 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/html 46400593 4 -r-x------ 1 yarn hadoop 937 Jun 7 12:11 ./sparkr/SparkR/html/00Index.html 46400592 4 -r-x------ 1 yarn hadoop 1319 Jun 7 12:11 ./sparkr/SparkR/html/R.css 46400628 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/R 46400630 24 -r-x------ 1 yarn hadoop 23836 Jun 7 12:11 ./sparkr/SparkR/R/SparkR.rdx 46400629 976 -r-x------ 1 yarn hadoop 999203 Jun 7 12:11 ./sparkr/SparkR/R/SparkR.rdb 46400631 4 -r-x------ 1 yarn hadoop 1056 Jun 7 12:11 ./sparkr/SparkR/R/SparkR 46400560 4 -r-x------ 1 yarn hadoop 1431 Jun 7 12:11 ./sparkr/SparkR/DESCRIPTION 46400622 4 drwx------ 2 yarn hadoop 4096 Jun 7 12:11 ./sparkr/SparkR/help 46400624 4 -r-x------ 1 yarn hadoop 64 Jun 7 12:11 ./sparkr/SparkR/help/paths.rds 46400627 4 -r-x------ 1 yarn hadoop 60 Jun 7 12:11 ./sparkr/SparkR/help/aliases.rds 46400625 4 -r-x------ 1 yarn hadoop 113 Jun 7 12:11 ./sparkr/SparkR/help/SparkR.rdx 46400623 0 -r-x------ 1 yarn hadoop 0 Jun 7 12:11 ./sparkr/SparkR/help/SparkR.rdb 46400626 0 -r-x------ 1 yarn hadoop 0 Jun 7 12:11 ./sparkr/SparkR/help/AnIndex 46400668 4 -rw-r--r-- 1 yarn hadoop 12 Jun 7 12:11 ./.container_tokens.crc 46400314 12 drwxr-xr-x 2 yarn hadoop 12288 Jun 7 12:10 ./__spark_libs__ 46400448 48 -r-xr-xr-x 1 yarn hadoop 46983 Apr 1 22:52 ./__spark_libs__/jackson-annotations-2.6.5.jar 46400414 164 -r-xr-xr-x 1 yarn hadoop 167421 Apr 1 22:52 ./__spark_libs__/jersey-client-2.22.2.jar 46400507 388 -r-xr-xr-x 1 yarn hadoop 395195 Apr 1 22:52 ./__spark_libs__/javolution-5.5.1.jar 46400420 2780 -r-xr-xr-x 1 yarn hadoop 2842667 Apr 1 22:52 ./__spark_libs__/bcprov-jdk15on-1.51.jar 46400541 44 -r-xr-xr-x 1 yarn hadoop 41263 Apr 1 22:52 ./__spark_libs__/jackson-module-paranamer-2.6.5.jar 46400542 52 -r-xr-xr-x 1 yarn hadoop 50619 Apr 1 22:52 ./__spark_libs__/chill-java-0.8.0.jar 46400360 164 -r-xr-xr-x 1 yarn hadoop 164368 Apr 1 22:52 ./__spark_libs__/antlr-runtime-3.4.jar 46400412 180 -r-xr-xr-x 1 yarn hadoop 181926 Apr 1 22:52 ./__spark_libs__/breeze-macros_2.11-0.12.jar 46400518 320 -r-xr-xr-x 1 yarn hadoop 325553 Apr 1 22:52 ./__spark_libs__/univocity-parsers-2.2.1.jar 46400483 8 -r-xr-xr-x 1 yarn hadoop 5950 Apr 1 22:52 ./__spark_libs__/javax.inject-2.4.0-b34.jar 46400514 212 -r-xr-xr-x 1 yarn hadoop 216419 Apr 1 22:52 ./__spark_libs__/hadoop-aws-2.7.3.2.6.0.3-8.jar 46400376 292 -r-xr-xr-x 1 yarn hadoop 298829 Apr 1 22:52 ./__spark_libs__/commons-configuration-1.6.jar 46400451 176 -r-xr-xr-x 1 yarn hadoop 178962 Apr 1 22:52 ./__spark_libs__/spark-mllib-local_2.11-2.1.0.2.6.0.3-8.jar 46400324 508 -r-xr-xr-x 1 yarn hadoop 516632 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-app-2.7.3.2.6.0.3-8.jar 46400511 200 -r-xr-xr-x 1 yarn hadoop 201928 Apr 1 22:52 ./__spark_libs__/RoaringBitmap-0.5.11.jar 46400477 176 -r-xr-xr-x 1 yarn hadoop 177214 Apr 1 22:52 ./__spark_libs__/jetty-util-6.1.26.hwx.jar 46400386 184 -r-xr-xr-x 1 yarn hadoop 185245 Apr 1 22:52 ./__spark_libs__/curator-framework-2.6.0.jar 46400479 428 -r-xr-xr-x 1 yarn hadoop 436303 Apr 1 22:52 ./__spark_libs__/avro-1.7.7.jar 46400380 7108 -r-xr-xr-x 1 yarn hadoop 7276083 Apr 1 22:52 ./__spark_libs__/spire_2.11-0.7.4.jar 46400382 15128 -r-xr-xr-x 1 yarn hadoop 15487351 Apr 1 22:52 ./__spark_libs__/scala-compiler-2.11.8.jar 46400508 144 -r-xr-xr-x 1 yarn hadoop 144660 Apr 1 22:52 ./__spark_libs__/xbean-asm5-shaded-4.4.jar 46400509 576 -r-xr-xr-x 1 yarn hadoop 589462 Apr 1 22:52 ./__spark_libs__/json4s-core_2.11-3.2.11.jar 46400472 296 -r-xr-xr-x 1 yarn hadoop 302248 Apr 1 22:52 ./__spark_libs__/antlr4-runtime-4.5.3.jar 46400487 1768 -r-xr-xr-x 1 yarn hadoop 1809447 Apr 1 22:52 ./__spark_libs__/datanucleus-rdbms-3.2.9.jar 46400366 232 -r-xr-xr-x 1 yarn hadoop 236880 Apr 1 22:52 ./__spark_libs__/lz4-1.3.0.jar 46400531 188 -r-xr-xr-x 1 yarn hadoop 188671 Apr 1 22:52 ./__spark_libs__/commons-beanutils-1.7.0.jar 46400481 116 -r-xr-xr-x 1 yarn hadoop 115534 Apr 1 22:52 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar 46400322 728 -r-xr-xr-x 1 yarn hadoop 745325 Apr 1 22:52 ./__spark_libs__/azure-storage-4.2.0.jar 46400370 212 -r-xr-xr-x 1 yarn hadoop 213911 Apr 1 22:52 ./__spark_libs__/jline-2.12.1.jar 46400484 684 -r-xr-xr-x 1 yarn hadoop 699286 Apr 1 22:52 ./__spark_libs__/spark-graphx_2.11-2.1.0.2.6.0.3-8.jar 46400468 48 -r-xr-xr-x 1 yarn hadoop 45210 Apr 1 22:52 ./__spark_libs__/spark-unsafe_2.11-2.1.0.2.6.0.3-8.jar 46400418 480 -r-xr-xr-x 1 yarn hadoop 489884 Apr 1 22:52 ./__spark_libs__/log4j-1.2.17.jar 46400537 1204 -r-xr-xr-x 1 yarn hadoop 1229125 Apr 1 22:52 ./__spark_libs__/xercesImpl-2.9.1.jar 46400437 11664 -r-xr-xr-x 1 yarn hadoop 11939986 Apr 1 22:52 ./__spark_libs__/spark-core_2.11-2.1.0.2.6.0.3-8.jar 46400543 1988 -r-xr-xr-x 1 yarn hadoop 2035066 Apr 1 22:52 ./__spark_libs__/commons-math3-3.4.1.jar 46400355 1500 -r-xr-xr-x 1 yarn hadoop 1535773 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-core-2.7.3.2.6.0.3-8.jar 46400352 32 -r-xr-xr-x 1 yarn hadoop 29555 Apr 1 22:52 ./__spark_libs__/paranamer-2.3.jar 46400425 100 -r-xr-xr-x 1 yarn hadoop 100682 Apr 1 22:52 ./__spark_libs__/hive-jdbc-1.2.1.spark2.hdp.jar 46400409 24 -r-xr-xr-x 1 yarn hadoop 20852 Apr 1 22:52 ./__spark_libs__/metrics-graphite-3.1.2.jar 46400503 1776 -r-xr-xr-x 1 yarn hadoop 1814993 Apr 1 22:52 ./__spark_libs__/spark-hive-thriftserver_2.11-2.1.0.2.6.0.3-8.jar 46400417 2732 -r-xr-xr-x 1 yarn hadoop 2796935 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-bundle-1.6.0.jar 46400501 280 -r-xr-xr-x 1 yarn hadoop 284184 Apr 1 22:52 ./__spark_libs__/commons-codec-1.10.jar 46400497 1208 -r-xr-xr-x 1 yarn hadoop 1232993 Apr 1 22:52 ./__spark_libs__/spark-hive_2.11-2.1.0.2.6.0.3-8.jar 46400426 20 -r-xr-xr-x 1 yarn hadoop 18899 Apr 1 22:52 ./__spark_libs__/jetty-sslengine-6.1.26.hwx.jar 46400529 764 -r-xr-xr-x 1 yarn hadoop 780664 Apr 1 22:52 ./__spark_libs__/jackson-mapper-asl-1.9.13.jar 46400456 64 -r-xr-xr-x 1 yarn hadoop 63777 Apr 1 22:52 ./__spark_libs__/validation-api-1.1.0.Final.jar 46400500 436 -r-xr-xr-x 1 yarn hadoop 442406 Apr 1 22:52 ./__spark_libs__/calcite-linq4j-1.2.0-incubating.jar 46400457 720 -r-xr-xr-x 1 yarn hadoop 736658 Apr 1 22:52 ./__spark_libs__/httpclient-4.5.2.jar 46400398 256 -r-xr-xr-x 1 yarn hadoop 258876 Apr 1 22:52 ./__spark_libs__/jackson-core-2.6.5.jar 46400326 1256 -r-xr-xr-x 1 yarn hadoop 1282424 Apr 1 22:52 ./__spark_libs__/ivy-2.4.0.jar 46400432 96 -r-xr-xr-x 1 yarn hadoop 96221 Apr 1 22:52 ./__spark_libs__/commons-pool-1.5.4.jar 46400335 44 -r-xr-xr-x 1 yarn hadoop 44925 Apr 1 22:52 ./__spark_libs__/apacheds-i18n-2.0.0-M15.jar 46400390 68 -r-xr-xr-x 1 yarn hadoop 65840 Apr 1 22:52 ./__spark_libs__/spark-repl_2.11-2.1.0.2.6.0.3-8.jar 46400532 144 -r-xr-xr-x 1 yarn hadoop 143602 Apr 1 22:52 ./__spark_libs__/commons-digester-1.8.jar 46400510 576 -r-xr-xr-x 1 yarn hadoop 588337 Apr 1 22:52 ./__spark_libs__/commons-collections-3.2.2.jar 46400449 48 -r-xr-xr-x 1 yarn hadoop 48720 Apr 1 22:52 ./__spark_libs__/snappy-0.2.jar 46400372 132 -r-xr-xr-x 1 yarn hadoop 134595 Apr 1 22:52 ./__spark_libs__/commons-crypto-1.0.0.jar 46400415 204 -r-xr-xr-x 1 yarn hadoop 206035 Apr 1 22:52 ./__spark_libs__/commons-beanutils-core-1.8.0.jar 46400365 56 -r-xr-xr-x 1 yarn hadoop 54071 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-shuffle-2.7.3.2.6.0.3-8.jar 46400351 20 -r-xr-xr-x 1 yarn hadoop 18336 Apr 1 22:52 ./__spark_libs__/jackson-jaxrs-1.9.13.jar 46400528 172 -r-xr-xr-x 1 yarn hadoop 174351 Apr 1 22:52 ./__spark_libs__/stream-2.7.0.jar 46400422 32 -r-xr-xr-x 1 yarn hadoop 29757 Apr 1 22:52 ./__spark_libs__/spark-sketch_2.11-2.1.0.2.6.0.3-8.jar 46400332 20 -r-xr-xr-x 1 yarn hadoop 20235 Apr 1 22:52 ./__spark_libs__/osgi-resource-locator-1.0.1.jar 46400499 3152 -r-xr-xr-x 1 yarn hadoop 3224708 Apr 1 22:52 ./__spark_libs__/derby-10.12.1.1.jar 46400354 84 -r-xr-xr-x 1 yarn hadoop 82421 Apr 1 22:52 ./__spark_libs__/json4s-ast_2.11-3.2.11.jar 46400423 748 -r-xr-xr-x 1 yarn hadoop 764569 Apr 1 22:52 ./__spark_libs__/jtransforms-2.4.0.jar 46400411 528 -r-xr-xr-x 1 yarn hadoop 540298 Apr 1 22:52 ./__spark_libs__/jetty-6.1.26.hwx.jar 46400444 16 -r-xr-xr-x 1 yarn hadoop 15071 Apr 1 22:52 ./__spark_libs__/jta-1.1.jar 46400379 20 -r-xr-xr-x 1 yarn hadoop 16560 Apr 1 22:52 ./__spark_libs__/api-asn1-api-1.0.0-M20.jar 46400328 16 -r-xr-xr-x 1 yarn hadoop 14766 Apr 1 22:52 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar 46400347 64 -r-xr-xr-x 1 yarn hadoop 65012 Apr 1 22:52 ./__spark_libs__/guice-servlet-3.0.jar 46400395 420 -r-xr-xr-x 1 yarn hadoop 427780 Apr 1 22:52 ./__spark_libs__/jodd-core-3.5.2.jar 46400428 640 -r-xr-xr-x 1 yarn hadoop 654216 Apr 1 22:52 ./__spark_libs__/pmml-model-1.2.15.jar 46400533 1024 -r-xr-xr-x 1 yarn hadoop 1048117 Apr 1 22:52 ./__spark_libs__/parquet-jackson-1.8.1.jar 46400357 684 -r-xr-xr-x 1 yarn hadoop 698375 Apr 1 22:52 ./__spark_libs__/jersey-common-2.22.2.jar 46400446 1848 -r-xr-xr-x 1 yarn hadoop 1890075 Apr 1 22:52 ./__spark_libs__/datanucleus-core-3.2.10.jar 46400358 7328 -r-xr-xr-x 1 yarn hadoop 7503383 Apr 1 22:52 ./__spark_libs__/spark-catalyst_2.11-2.1.0.2.6.0.3-8.jar 46400459 116 -r-xr-xr-x 1 yarn hadoop 114746 Apr 1 22:52 ./__spark_libs__/hadoop-openstack-2.7.3.2.6.0.3-8.jar 46400434 164 -r-xr-xr-x 1 yarn hadoop 164422 Apr 1 22:52 ./__spark_libs__/core-1.1.2.jar 46400489 192 -r-xr-xr-x 1 yarn hadoop 192993 Apr 1 22:52 ./__spark_libs__/avro-ipc-1.7.7.jar 46400388 212 -r-xr-xr-x 1 yarn hadoop 213786 Apr 1 22:52 ./__spark_libs__/hadoop-azure-2.7.3.2.6.0.3-8.jar 46400323 1168 -r-xr-xr-x 1 yarn hadoop 1194003 Apr 1 22:52 ./__spark_libs__/arpack_combined_all-0.1.jar 46400410 1444 -r-xr-xr-x 1 yarn hadoop 1475955 Apr 1 22:52 ./__spark_libs__/htrace-core-3.1.0-incubating.jar 46400513 64 -r-xr-xr-x 1 yarn hadoop 64661 Apr 1 22:52 ./__spark_libs__/okio-1.4.0.jar 46400462 1144 -r-xr-xr-x 1 yarn hadoop 1171380 Apr 1 22:52 ./__spark_libs__/jackson-databind-2.6.5.jar 46400329 52 -r-xr-xr-x 1 yarn hadoop 51484 Apr 1 22:52 ./__spark_libs__/json-smart-1.1.1.jar 46400344 2304 -r-xr-xr-x 1 yarn hadoop 2356793 Apr 1 22:52 ./__spark_libs__/spark-network-common_2.11-2.1.0.2.6.0.3-8.jar 46400538 952 -r-xr-xr-x 1 yarn hadoop 971310 Apr 1 22:52 ./__spark_libs__/jersey-guava-2.22.2.jar 46400367 560 -r-xr-xr-x 1 yarn hadoop 570101 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-s3-1.10.6.jar 46400441 180 -r-xr-xr-x 1 yarn hadoop 181271 Apr 1 22:52 ./__spark_libs__/hk2-locator-2.4.0-b34.jar 46400512 472 -r-xr-xr-x 1 yarn hadoop 479881 Apr 1 22:52 ./__spark_libs__/commons-lang3-3.5.jar 46400340 180 -r-xr-xr-x 1 yarn hadoop 180736 Apr 1 22:52 ./__spark_libs__/avro-mapred-1.7.7-hadoop2.jar 46400333 280 -r-xr-xr-x 1 yarn hadoop 285479 Apr 1 22:52 ./__spark_libs__/parquet-encoding-1.8.1.jar 46400374 14080 -r-xr-xr-x 1 yarn hadoop 14413834 Apr 1 22:52 ./__spark_libs__/breeze_2.11-0.12.jar 46400378 20 -r-xr-xr-x 1 yarn hadoop 16430 Apr 1 22:52 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar 46400526 220 -r-xr-xr-x 1 yarn hadoop 223573 Apr 1 22:52 ./__spark_libs__/chill_2.11-0.8.0.jar 46400463 68 -r-xr-xr-x 1 yarn hadoop 68866 Apr 1 22:52 ./__spark_libs__/curator-client-2.6.0.jar 46400421 1996 -r-xr-xr-x 1 yarn hadoop 2041628 Apr 1 22:52 ./__spark_libs__/jets3t-0.9.3.jar 46400339 28 -r-xr-xr-x 1 yarn hadoop 26514 Apr 1 22:52 ./__spark_libs__/stax-api-1.0.1.jar 46400342 504 -r-xr-xr-x 1 yarn hadoop 515604 Apr 1 22:52 ./__spark_libs__/jackson-module-scala_2.11-2.6.5.jar 46400492 788 -r-xr-xr-x 1 yarn hadoop 802818 Apr 1 22:52 ./__spark_libs__/scalap-2.11.8.jar 46400522 84 -r-xr-xr-x 1 yarn hadoop 82908 Apr 1 22:52 ./__spark_libs__/spark-cloud_2.11-2.1.0.2.6.0.3-8.jar 46400516 116 -r-xr-xr-x 1 yarn hadoop 115472 Apr 1 22:52 ./__spark_libs__/py4j-0.10.4.jar 46400377 20 -r-xr-xr-x 1 yarn hadoop 19827 Apr 1 22:52 ./__spark_libs__/opencsv-2.3.jar 46400368 6048 -r-xr-xr-x 1 yarn hadoop 6189401 Apr 1 22:52 ./__spark_libs__/spark-mllib_2.11-2.1.0.2.6.0.3-8.jar 46400413 24 -r-xr-xr-x 1 yarn hadoop 23346 Apr 1 22:52 ./__spark_libs__/stax-api-1.0-2.jar 46400502 44 -r-xr-xr-x 1 yarn hadoop 41755 Apr 1 22:52 ./__spark_libs__/objenesis-2.1.jar 46400524 68 -r-xr-xr-x 1 yarn hadoop 69409 Apr 1 22:52 ./__spark_libs__/activation-1.1.1.jar 46400405 380 -r-xr-xr-x 1 yarn hadoop 387188 Apr 1 22:52 ./__spark_libs__/parquet-format-2.3.0-incubating.jar 46400383 8 -r-xr-xr-x 1 yarn hadoop 5310 Apr 1 22:52 ./__spark_libs__/pmml-schema-1.2.15.jar 46400427 176 -r-xr-xr-x 1 yarn hadoop 178648 Apr 1 22:52 ./__spark_libs__/nimbus-jose-jwt-3.9.jar 46400485 40 -r-xr-xr-x 1 yarn hadoop 39280 Apr 1 22:52 ./__spark_libs__/metrics-jvm-3.1.2.jar 46400475 20 -r-xr-xr-x 1 yarn hadoop 18098 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-2.22.2.jar 46400520 720 -r-xr-xr-x 1 yarn hadoop 735574 Apr 1 22:52 ./__spark_libs__/janino-3.0.0.jar 46400431 636 -r-xr-xr-x 1 yarn hadoop 648678 Apr 1 22:52 ./__spark_libs__/scala-xml_2.11-1.0.2.jar 46400403 776 -r-xr-xr-x 1 yarn hadoop 794543 Apr 1 22:52 ./__spark_libs__/zookeeper-3.4.6.2.6.0.3-8.jar 46400336 12 -r-xr-xr-x 1 yarn hadoop 10092 Apr 1 22:52 ./__spark_libs__/azure-keyvault-core-0.8.0.jar 46400392 512 -r-xr-xr-x 1 yarn hadoop 521157 Apr 1 22:52 ./__spark_libs__/mail-1.4.7.jar 46400356 96 -r-xr-xr-x 1 yarn hadoop 94672 Apr 1 22:52 ./__spark_libs__/xz-1.0.jar 46400406 80 -r-xr-xr-x 1 yarn hadoop 79173 Apr 1 22:52 ./__spark_libs__/hadoop-auth-2.7.3.2.6.0.3-8.jar 46400364 8 -r-xr-xr-x 1 yarn hadoop 5711 Apr 1 22:52 ./__spark_libs__/minlog-1.3.0.jar 46400402 3564 -r-xr-xr-x 1 yarn hadoop 3646287 Apr 1 22:52 ./__spark_libs__/hadoop-common-2.7.3.2.6.0.3-8.jar 46400385 400 -r-xr-xr-x 1 yarn hadoop 409467 Apr 1 22:52 ./__spark_libs__/mx4j-3.0.2.jar 46400439 700 -r-xr-xr-x 1 yarn hadoop 714194 Apr 1 22:52 ./__spark_libs__/javassist-3.18.1-GA.jar 46400350 932 -r-xr-xr-x 1 yarn hadoop 951701 Apr 1 22:52 ./__spark_libs__/jersey-server-2.22.2.jar 46400465 308 -r-xr-xr-x 1 yarn hadoop 313702 Apr 1 22:52 ./__spark_libs__/libfb303-0.9.3.jar 46400430 16 -r-xr-xr-x 1 yarn hadoop 15010 Apr 1 22:52 ./__spark_libs__/xmlenc-0.52.jar 46400482 12 -r-xr-xr-x 1 yarn hadoop 10023 Apr 1 22:52 ./__spark_libs__/java-xmlbuilder-1.0.jar 46400359 80 -r-xr-xr-x 1 yarn hadoop 78849 Apr 1 22:52 ./__spark_libs__/azure-data-lake-store-sdk-2.1.4.jar 46400384 20 -r-xr-xr-x 1 yarn hadoop 17008 Apr 1 22:52 ./__spark_libs__/base64-2.3.8.jar 46400478 72 -r-xr-xr-x 1 yarn hadoop 72733 Apr 1 22:52 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar 46400404 728 -r-xr-xr-x 1 yarn hadoop 742988 Apr 1 22:52 ./__spark_libs__/spark-yarn_2.11-2.1.0.2.6.0.3-8.jar 46400464 64 -r-xr-xr-x 1 yarn hadoop 61837 Apr 1 22:52 ./__spark_libs__/spark-network-shuffle_2.11-2.1.0.2.6.0.3-8.jar 46400393 16 -r-xr-xr-x 1 yarn hadoop 15523 Apr 1 22:52 ./__spark_libs__/spark-tags_2.11-2.1.0.2.6.0.3-8.jar 46400505 4 -r-xr-xr-x 1 yarn hadoop 2497 Apr 1 22:52 ./__spark_libs__/javax.inject-1.jar 46400361 112 -r-xr-xr-x 1 yarn hadoop 112558 Apr 1 22:52 ./__spark_libs__/metrics-core-3.1.2.jar 46400424 40 -r-xr-xr-x 1 yarn hadoop 40509 Apr 1 22:52 ./__spark_libs__/slf4j-api-1.7.16.jar 46400371 64 -r-xr-xr-x 1 yarn hadoop 65261 Apr 1 22:52 ./__spark_libs__/oro-2.0.8.jar 46400334 120 -r-xr-xr-x 1 yarn hadoop 118973 Apr 1 22:52 ./__spark_libs__/hk2-utils-2.4.0-b34.jar 46400400 8 -r-xr-xr-x 1 yarn hadoop 4596 Apr 1 22:52 ./__spark_libs__/jul-to-slf4j-1.7.16.jar 46400327 40 -r-xr-xr-x 1 yarn hadoop 38604 Apr 1 22:52 ./__spark_libs__/parquet-common-1.8.1.jar 46400346 300 -r-xr-xr-x 1 yarn hadoop 305001 Apr 1 22:52 ./__spark_libs__/commons-httpclient-3.1.jar 46400330 28 -r-xr-xr-x 1 yarn hadoop 26366 Apr 1 22:52 ./__spark_libs__/javax.annotation-api-1.2.jar 46400545 68 -r-xr-xr-x 1 yarn hadoop 66270 Apr 1 22:52 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar 46400396 4 -r-xr-xr-x 1 yarn hadoop 2254 Apr 1 22:52 ./__spark_libs__/jcip-annotations-1.0.jar 46400375 20 -r-xr-xr-x 1 yarn hadoop 18482 Apr 1 22:52 ./__spark_libs__/eigenbase-properties-1.1.5.jar 46400539 232 -r-xr-xr-x 1 yarn hadoop 234201 Apr 1 22:52 ./__spark_libs__/libthrift-0.9.3.jar 46400523 2052 -r-xr-xr-x 1 yarn hadoop 2101079 Apr 1 22:52 ./__spark_libs__/spark-streaming_2.11-2.1.0.2.6.0.3-8.jar 46400454 80 -r-xr-xr-x 1 yarn hadoop 79845 Apr 1 22:52 ./__spark_libs__/compress-lzf-1.0.3.jar 46400535 1032 -r-xr-xr-x 1 yarn hadoop 1056168 Apr 1 22:52 ./__spark_libs__/snappy-java-1.1.2.6.jar 46400401 1204 -r-xr-xr-x 1 yarn hadoop 1230201 Apr 1 22:52 ./__spark_libs__/netty-3.8.0.Final.jar 46400452 208 -r-xr-xr-x 1 yarn hadoop 212453 Apr 1 22:52 ./__spark_libs__/commons-net-2.2.jar 46400436 112 -r-xr-xr-x 1 yarn hadoop 110600 Apr 1 22:52 ./__spark_libs__/bonecp-0.8.0.RELEASE.jar 46400455 40 -r-xr-xr-x 1 yarn hadoop 40856 Apr 1 22:52 ./__spark_libs__/hive-cli-1.2.1.spark2.hdp.jar 46400348 36 -r-xr-xr-x 1 yarn hadoop 36247 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-web-proxy-2.7.3.2.6.0.3-8.jar 46400399 148 -r-xr-xr-x 1 yarn hadoop 148627 Apr 1 22:52 ./__spark_libs__/stringtemplate-3.2.1.jar 46400544 352 -r-xr-xr-x 1 yarn hadoop 358390 Apr 1 22:52 ./__spark_libs__/kryo-shaded-3.0.3.jar 46400467 20 -r-xr-xr-x 1 yarn hadoop 16993 Apr 1 22:52 ./__spark_libs__/JavaEWAH-0.3.2.jar 46400534 524 -r-xr-xr-x 1 yarn hadoop 533455 Apr 1 22:52 ./__spark_libs__/protobuf-java-2.5.0.jar 46400460 92 -r-xr-xr-x 1 yarn hadoop 93210 Apr 1 22:52 ./__spark_libs__/super-csv-2.2.0.jar 46400473 192 -r-xr-xr-x 1 yarn hadoop 193238 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-client-2.7.3.2.6.0.3-8.jar 46400506 4 -r-xr-xr-x 1 yarn hadoop 2566 Apr 1 22:52 ./__spark_libs__/hadoop-client-2.7.3.2.6.0.3-8.jar 46400445 440 -r-xr-xr-x 1 yarn hadoop 448794 Apr 1 22:52 ./__spark_libs__/apache-log4j-extras-1.2.17.jar 46400540 256 -r-xr-xr-x 1 yarn hadoop 258370 Apr 1 22:52 ./__spark_libs__/calcite-avatica-1.2.0-incubating.jar 46400504 232 -r-xr-xr-x 1 yarn hadoop 236660 Apr 1 22:52 ./__spark_libs__/ST4-4.0.4.jar 46400429 2264 -r-xr-xr-x 1 yarn hadoop 2314839 Apr 1 22:52 ./__spark_libs__/netty-all-4.0.42.Final.jar 46400416 88 -r-xr-xr-x 1 yarn hadoop 86811 Apr 1 22:52 ./__spark_libs__/spire-macros_2.11-0.7.4.jar 46400325 676 -r-xr-xr-x 1 yarn hadoop 691479 Apr 1 22:52 ./__spark_libs__/apacheds-kerberos-codec-2.0.0-M15.jar 46400381 2068 -r-xr-xr-x 1 yarn hadoop 2117552 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-api-2.7.3.2.6.0.3-8.jar 46400480 216 -r-xr-xr-x 1 yarn hadoop 218076 Apr 1 22:52 ./__spark_libs__/parquet-hadoop-1.8.1.jar 46400519 12 -r-xr-xr-x 1 yarn hadoop 9939 Apr 1 22:52 ./__spark_libs__/slf4j-log4j12-1.7.16.jar 46400447 24 -r-xr-xr-x 1 yarn hadoop 23838 Apr 1 22:52 ./__spark_libs__/hadoop-azure-datalake-2.7.3.2.6.0.3-8.jar 46400498 3440 -r-xr-xr-x 1 yarn hadoop 3519262 Apr 1 22:52 ./__spark_libs__/calcite-core-1.2.0-incubating.jar 46400525 200 -r-xr-xr-x 1 yarn hadoop 201124 Apr 1 22:52 ./__spark_libs__/jdo-api-3.0.1.jar 46400461 6504 -r-xr-xr-x 1 yarn hadoop 6657956 Apr 1 22:52 ./__spark_libs__/spark-sql_2.11-2.1.0.2.6.0.3-8.jar 46400471 104 -r-xr-xr-x 1 yarn hadoop 105134 Apr 1 22:52 ./__spark_libs__/jaxb-api-2.2.2.jar 46400343 244 -r-xr-xr-x 1 yarn hadoop 248171 Apr 1 22:52 ./__spark_libs__/curator-recipes-2.6.0.jar 46400466 1024 -r-xr-xr-x 1 yarn hadoop 1045744 Apr 1 22:52 ./__spark_libs__/leveldbjni-all-1.8.jar 46400442 616 -r-xr-xr-x 1 yarn hadoop 627814 Apr 1 22:52 ./__spark_libs__/joda-time-2.9.3.jar 46400458 96 -r-xr-xr-x 1 yarn hadoop 95806 Apr 1 22:52 ./__spark_libs__/javax.servlet-api-3.1.0.jar 46400407 412 -r-xr-xr-x 1 yarn hadoop 420785 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-server-common-2.7.3.2.6.0.3-8.jar 46400337 696 -r-xr-xr-x 1 yarn hadoop 710492 Apr 1 22:52 ./__spark_libs__/guice-3.0.jar 46400474 256 -r-xr-xr-x 1 yarn hadoop 258578 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-kms-1.10.6.jar 46400488 8 -r-xr-xr-x 1 yarn hadoop 4467 Apr 1 22:52 ./__spark_libs__/aopalliance-1.0.jar 46400470 64 -r-xr-xr-x 1 yarn hadoop 62050 Apr 1 22:52 ./__spark_libs__/commons-logging-1.1.3.jar 46400440 5612 -r-xr-xr-x 1 yarn hadoop 5744974 Apr 1 22:52 ./__spark_libs__/scala-library-2.11.8.jar 46400338 184 -r-xr-xr-x 1 yarn hadoop 185140 Apr 1 22:52 ./__spark_libs__/commons-io-2.4.jar 46400408 28 -r-xr-xr-x 1 yarn hadoop 27084 Apr 1 22:52 ./__spark_libs__/jackson-xc-1.9.13.jar 46400493 12 -r-xr-xr-x 1 yarn hadoop 12131 Apr 1 22:52 ./__spark_libs__/jpam-1.1.jar 46400530 924 -r-xr-xr-x 1 yarn hadoop 945914 Apr 1 22:52 ./__spark_libs__/parquet-column-1.8.1.jar 46400362 332 -r-xr-xr-x 1 yarn hadoop 339666 Apr 1 22:52 ./__spark_libs__/datanucleus-api-jdo-3.2.6.jar 46400341 136 -r-xr-xr-x 1 yarn hadoop 138449 Apr 1 22:52 ./__spark_libs__/hive-beeline-1.2.1.spark2.hdp.jar 46400521 100 -r-xr-xr-x 1 yarn hadoop 101125 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-registry-2.7.3.2.6.0.3-8.jar 46400491 40 -r-xr-xr-x 1 yarn hadoop 40341 Apr 1 22:52 ./__spark_libs__/json4s-jackson_2.11-3.2.11.jar 46400435 16 -r-xr-xr-x 1 yarn hadoop 15827 Apr 1 22:52 ./__spark_libs__/metrics-json-3.1.2.jar 46400369 36 -r-xr-xr-x 1 yarn hadoop 36627 Apr 1 22:52 ./__spark_libs__/commons-compiler-3.0.0.jar 46400496 1712 -r-xr-xr-x 1 yarn hadoop 1750519 Apr 1 22:52 ./__spark_libs__/hadoop-yarn-common-2.7.3.2.6.0.3-8.jar 46400391 80 -r-xr-xr-x 1 yarn hadoop 79912 Apr 1 22:52 ./__spark_libs__/api-util-1.0.0-M20.jar 46400331 176 -r-xr-xr-x 1 yarn hadoop 178947 Apr 1 22:52 ./__spark_libs__/hk2-api-2.4.0-b34.jar 46400469 40 -r-xr-xr-x 1 yarn hadoop 39395 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-jobclient-2.7.3.2.6.0.3-8.jar 46400387 20 -r-xr-xr-x 1 yarn hadoop 17369 Apr 1 22:52 ./__spark_libs__/hadoop-annotations-2.7.3.2.6.0.3-8.jar 46400397 100 -r-xr-xr-x 1 yarn hadoop 100636 Apr 1 22:52 ./__spark_libs__/jsp-api-2.1.jar 46400419 11232 -r-xr-xr-x 1 yarn hadoop 11499118 Apr 1 22:52 ./__spark_libs__/hive-exec-1.2.1.spark2.hdp.jar 46400349 2140 -r-xr-xr-x 1 yarn hadoop 2189117 Apr 1 22:52 ./__spark_libs__/guava-14.0.1.jar 46400494 48 -r-xr-xr-x 1 yarn hadoop 48418 Apr 1 22:52 ./__spark_libs__/jackson-dataformat-cbor-2.6.5.jar 46400515 228 -r-xr-xr-x 1 yarn hadoop 232248 Apr 1 22:52 ./__spark_libs__/jackson-core-asl-1.9.13.jar 46400476 236 -r-xr-xr-x 1 yarn hadoop 241367 Apr 1 22:52 ./__spark_libs__/commons-compress-1.4.1.jar 46400495 2004 -r-xr-xr-x 1 yarn hadoop 2049451 Apr 1 22:52 ./__spark_libs__/shapeless_2.11-2.0.0.jar 46400453 160 -r-xr-xr-x 1 yarn hadoop 160519 Apr 1 22:52 ./__spark_libs__/commons-dbcp-1.4.jar 46400373 320 -r-xr-xr-x 1 yarn hadoop 326724 Apr 1 22:52 ./__spark_libs__/httpcore-4.4.4.jar 46400438 72 -r-xr-xr-x 1 yarn hadoop 70589 Apr 1 22:52 ./__spark_libs__/spark-launcher_2.11-2.1.0.2.6.0.3-8.jar 46400443 44 -r-xr-xr-x 1 yarn hadoop 41123 Apr 1 22:52 ./__spark_libs__/commons-cli-1.2.jar 46400450 5380 -r-xr-xr-x 1 yarn hadoop 5505067 Apr 1 22:52 ./__spark_libs__/hive-metastore-1.2.1.spark2.hdp.jar 46400345 8484 -r-xr-xr-x 1 yarn hadoop 8686273 Apr 1 22:52 ./__spark_libs__/hadoop-hdfs-2.7.3.2.6.0.3-8.jar 46400486 436 -r-xr-xr-x 1 yarn hadoop 445288 Apr 1 22:52 ./__spark_libs__/antlr-2.7.7.jar 46400517 280 -r-xr-xr-x 1 yarn hadoop 284220 Apr 1 22:52 ./__spark_libs__/commons-lang-2.6.jar 46400389 96 -r-xr-xr-x 1 yarn hadoop 94796 Apr 1 22:52 ./__spark_libs__/pyrolite-4.13.jar 46400527 312 -r-xr-xr-x 1 yarn hadoop 319099 Apr 1 22:52 ./__spark_libs__/okhttp-2.4.0.jar 46400363 736 -r-xr-xr-x 1 yarn hadoop 752751 Apr 1 22:52 ./__spark_libs__/hadoop-mapreduce-client-common-2.7.3.2.6.0.3-8.jar 46400536 416 -r-xr-xr-x 1 yarn hadoop 423753 Apr 1 22:52 ./__spark_libs__/scala-parser-combinators_2.11-1.0.4.jar 46400394 504 -r-xr-xr-x 1 yarn hadoop 516062 Apr 1 22:52 ./__spark_libs__/aws-java-sdk-core-1.10.6.jar 46400433 36 -r-xr-xr-x 1 yarn hadoop 33015 Apr 1 22:52 ./__spark_libs__/jsr305-1.3.9.jar 46400353 4468 -r-xr-xr-x 1 yarn hadoop 4573750 Apr 1 22:52 ./__spark_libs__/scala-reflect-2.11.8.jar 46400490 188 -r-xr-xr-x 1 yarn hadoop 190432 Apr 1 22:52 ./__spark_libs__/gson-2.2.4.jar 115606291 76 -r-x------ 1 yarn hadoop 74096 Jun 7 12:11 ./py4j-0.10.4-src.zip broken symlinks(find -L . -maxdepth 5 -type l -ls): End of LogType:directory.info LogType:launch_container.sh Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:6655 Log Contents: #!/bin/bash export SPARK_YARN_STAGING_DIR="hdfs://hdfs-hadoop-88hh.c.creditvidya-152512.internal:8020/user/zeppelin/.sparkStaging/application_1496837077102_0003" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/current/hadoop-client/conf"} export JAVA_HOME=${JAVA_HOME:-"/usr/jdk64/jdk1.8.0_112"} export SPARK_LOG_URL_STDOUT="http://hdfs-hadoop-p0z7.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000003/zeppelin/stdout?start=-4096" export NM_HOST="hdfs-hadoop-p0z7.c.creditvidya-152512.internal" export LOGNAME="zeppelin" export JVM_PID="$$" export PWD="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003" export LOCAL_DIRS="/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003,/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003" export PYTHONPATH="/usr/hdp/current/spark2-client//python/lib/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python/:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:$PWD/pyspark.zip:$PWD/py4j-0.10.4-src.zip:/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export NM_HTTP_PORT="8042" export LOG_DIRS="/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003,/hadoop/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= " export NM_PORT="45454" export USER="zeppelin" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/current/hadoop-yarn-nodemanager"} export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:/etc/hadoop/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hadoop-client/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.6.0.3-8/hadoop/lib/hadoop-lzo-0.6.0.2.6.0.3-8.jar:/etc/hadoop/conf/secure" export SPARK_YARN_MODE="true" export HADOOP_TOKEN_FILE_LOCATION="/hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/container_tokens" export NM_AUX_SERVICE_spark_shuffle="" export SPARK_USER="zeppelin" export LOCAL_USER_DIRS="/hadoop/yarn/local/usercache/zeppelin/,/hadoop/hadoop/yarn/local/usercache/zeppelin/" export SPARK_YARN_USER_ENV="PYTHONPATH=/usr/hdp/current/spark2-client//python:/usr/hdp/current/spark2-client//python/lib/py4j-0.8.2.1-src.zip" export SPARK_LOG_URL_STDERR="http://hdfs-hadoop-p0z7.c.creditvidya-152512.internal:8042/node/containerlogs/container_e02_1496837077102_0003_01_000003/zeppelin/stderr?start=-4096" export HOME="/home/" export NM_AUX_SERVICE_spark2_shuffle="" export CONTAINER_ID="container_e02_1496837077102_0003_01_000003" export MALLOC_ARENA_MAX="4" ln -sf "/hadoop/hadoop/yarn/local/filecache/10/spark2-hdp-yarn-archive.tar.gz" "__spark_libs__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/35/pyspark.zip" "pyspark.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/36/__spark_conf__.zip" "__spark_conf__" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/yarn/local/usercache/zeppelin/filecache/37/py4j-0.10.4-src.zip" "py4j-0.10.4-src.zip" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/hadoop/hadoop/yarn/local/usercache/zeppelin/filecache/34/sparkr.zip" "sparkr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi # Creating copy of launch script cp "launch_container.sh" "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/launch_container.sh" chmod 640 "/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/launch_container.sh" # Determining directory contents echo "ls -l:" 1>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/directory.info" ls -l 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/directory.info" echo "find -L . -maxdepth 5 -ls:" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/directory.info" find -L . -maxdepth 5 -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/directory.info" echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/directory.info" find -L . -maxdepth 5 -type l -ls 1>>"/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/directory.info" exec /bin/bash -c "LD_LIBRARY_PATH="/usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:$LD_LIBRARY_PATH" $JAVA_HOME/bin/java -server -Xmx1024m -Djava.io.tmpdir=$PWD/tmp '-Dspark.history.ui.port=18081' '-Dspark.driver.port=45013' -Dspark.yarn.app.container.log.dir=/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003 -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.141.0.9:45013 --executor-id 2 --hostname hdfs-hadoop-p0z7.c.creditvidya-152512.internal --cores 1 --app-id application_1496837077102_0003 --user-class-path file:$PWD/__app__.jar 1>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/stdout 2>/hadoop/yarn/log/application_1496837077102_0003/container_e02_1496837077102_0003_01_000003/stderr" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi End of LogType:launch_container.sh LogType:stderr Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:4510 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/hadoop/yarn/local/filecache/10/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/06/07 12:11:35 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 23200@hdfs-hadoop-p0z7 17/06/07 12:11:35 INFO SignalUtils: Registered signal handler for TERM 17/06/07 12:11:35 INFO SignalUtils: Registered signal handler for HUP 17/06/07 12:11:35 INFO SignalUtils: Registered signal handler for INT 17/06/07 12:11:35 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 12:11:35 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 12:11:35 INFO SecurityManager: Changing view acls groups to: 17/06/07 12:11:35 INFO SecurityManager: Changing modify acls groups to: 17/06/07 12:11:35 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 12:11:36 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 75 ms (0 ms spent in bootstraps) 17/06/07 12:11:36 INFO SecurityManager: Changing view acls to: yarn,zeppelin 17/06/07 12:11:36 INFO SecurityManager: Changing modify acls to: yarn,zeppelin 17/06/07 12:11:36 INFO SecurityManager: Changing view acls groups to: 17/06/07 12:11:36 INFO SecurityManager: Changing modify acls groups to: 17/06/07 12:11:36 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, zeppelin); groups with view permissions: Set(); users with modify permissions: Set(yarn, zeppelin); groups with modify permissions: Set() 17/06/07 12:11:36 INFO TransportClientFactory: Successfully created connection to /10.141.0.9:45013 after 1 ms (0 ms spent in bootstraps) 17/06/07 12:11:36 INFO DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-d96588da-0fb7-4c1a-998a-a197c8d9c1e8 17/06/07 12:11:36 INFO DiskBlockManager: Created local directory at /hadoop/hadoop/yarn/local/usercache/zeppelin/appcache/application_1496837077102_0003/blockmgr-6c3f9d2b-60cd-4723-802f-19f124822aa4 17/06/07 12:11:36 INFO MemoryStore: MemoryStore started with capacity 366.3 MB 17/06/07 12:11:36 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@10.141.0.9:45013 17/06/07 12:11:36 INFO CoarseGrainedExecutorBackend: Successfully registered with driver 17/06/07 12:11:36 INFO Executor: Starting executor ID 2 on host hdfs-hadoop-p0z7.c.creditvidya-152512.internal 17/06/07 12:11:36 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 45109. 17/06/07 12:11:36 INFO NettyBlockTransferService: Server created on hdfs-hadoop-p0z7.c.creditvidya-152512.internal:45109 17/06/07 12:11:36 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 17/06/07 12:11:36 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(2, hdfs-hadoop-p0z7.c.creditvidya-152512.internal, 45109, None) 17/06/07 12:11:36 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(2, hdfs-hadoop-p0z7.c.creditvidya-152512.internal, 45109, None) 17/06/07 12:11:36 INFO BlockManager: external shuffle service port = 7447 17/06/07 12:11:36 INFO BlockManager: Registering executor with local external shuffle service. 17/06/07 12:11:36 INFO TransportClientFactory: Successfully created connection to hdfs-hadoop-p0z7.c.creditvidya-152512.internal/10.141.0.9:7447 after 2 ms (0 ms spent in bootstraps) 17/06/07 12:11:36 INFO BlockManager: Initialized BlockManager: BlockManagerId(2, hdfs-hadoop-p0z7.c.creditvidya-152512.internal, 45109, None) 17/06/07 12:11:36 INFO Executor: Using REPL class URI: spark://10.141.0.9:45013/classes 17/06/07 12:12:38 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/06/07 12:12:38 INFO DiskBlockManager: Shutdown hook called 17/06/07 12:12:38 INFO ShutdownHookManager: Shutdown hook called End of LogType:stderr LogType:stdout Log Upload Time:Wed Jun 07 16:28:14 +0000 2017 LogLength:0 Log Contents: End of LogType:stdout