16/12/07 07:34:58 INFO InteractiveSession: Creating LivyClient for sessionId: 1 16/12/07 07:34:58 WARN RSCConf: Your hostname, rdeluh0001075.rz-c007-j650.basf-ag.de, resolves to a loopback address, but we couldn't find any external IP address! 16/12/07 07:34:58 WARN RSCConf: Set livy.rsc.rpc.server.address if you need to bind to another address. 16/12/07 07:34:58 INFO SessionManager: Registering new session 1 Dec 07, 2016 7:34:59 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:34:59 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Dec 07, 2016 7:35:00 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:00 INFO TimelineClientImpl: Timeline service address: http://rdeluh0001074.rz-c007-j650.basf-ag.de:8188/ws/v1/timeline/ Dec 07, 2016 7:35:01 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:01 INFO AHSProxy: Connecting to Application History server at rdeluh0001074.rz-c007-j650.basf-ag.de/10.4.126.15:10200 Dec 07, 2016 7:35:01 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:01 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 WARN Client: Failed to connect to server: rdeluh0001074.rz-c007-j650.basf-ag.de/10.4.126.15:8032: retries get failed due to exceeded maximum allowed retries number: 0 Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: java.net.ConnectException: Connection refused Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:650) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:745) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.Client$Connection.access$3200(Client.java:397) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.Client.getConnection(Client.java:1618) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.Client.call(Client.java:1449) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.Client.call(Client.java:1396) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at com.sun.proxy.$Proxy17.getClusterMetrics(Unknown Source) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:206) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at java.lang.reflect.Method.invoke(Method.java:498) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:278) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:194) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:176) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at com.sun.proxy.$Proxy18.getClusterMetrics(Unknown Source) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:506) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:130) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:130) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.Logging$class.logInfo(Logging.scala:58) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:62) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:129) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client.run(Client.scala:1109) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1169) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.yarn.Client.main(Client.scala) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at java.lang.reflect.Method.invoke(Method.java:498) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:163) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:161) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at java.security.AccessController.doPrivileged(Native Method) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at javax.security.auth.Subject.doAs(Subject.java:422) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:161) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO ConfiguredRMFailoverProxyProvider: Failing over to rm2 Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Requesting a new application from cluster with 3 NodeManagers Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (64512 MB per container) Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Will allocate AM container, with 896 MB memory including 384 MB overhead Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Setting up container launch context for our AM Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Setting up the launch environment for our AM container Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Using the spark assembly jar on HDFS because you are using HDP, defaultSparkAssembly:hdfs://HDPDDMDEV/hdp/apps/2.5.0.0-1245/spark/spark-hdp-assembly.jar Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO Client: Preparing resources for our AM container Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO YarnSparkHadoopUtil: getting token for namenode: hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004 Dec 07, 2016 7:35:02 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:02 INFO DFSClient: Created HDFS_DELEGATION_TOKEN token 924 for zeppelin-hdp_ddm_dev on ha-hdfs:HDPDDMDEV Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO metastore: Trying to connect to metastore with URI thrift://rdeluh0001074.rz-c007-j650.basf-ag.de:9083 Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO metastore: Connected to metastore. Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO YarnSparkHadoopUtil: HBase class not found java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Using the spark assembly jar on HDFS because you are using HDP, defaultSparkAssembly:hdfs://HDPDDMDEV/hdp/apps/2.5.0.0-1245/spark/spark-hdp-assembly.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Source and destination file systems are the same. Not copying hdfs://HDPDDMDEV/hdp/apps/2.5.0.0-1245/spark/spark-hdp-assembly.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/usr/hdp/current/livy-server/rsc-jars/netty-all-4.0.23.Final.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/netty-all-4.0.23.Final.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/usr/hdp/current/livy-server/rsc-jars/livy-api-0.2.0.2.5.0.0-1245.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/livy-api-0.2.0.2.5.0.0-1245.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/usr/hdp/current/livy-server/rsc-jars/livy-rsc-0.2.0.2.5.0.0-1245.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/livy-rsc-0.2.0.2.5.0.0-1245.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/usr/hdp/current/livy-server/repl-jars/commons-codec-1.9.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/commons-codec-1.9.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/usr/hdp/current/livy-server/repl-jars/livy-core-0.2.0.2.5.0.0-1245.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/livy-core-0.2.0.2.5.0.0-1245.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/usr/hdp/current/livy-server/repl-jars/livy-repl-0.2.0.2.5.0.0-1245.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/livy-repl-0.2.0.2.5.0.0-1245.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/hadoop/hdp/current/spark-client/lib/datanucleus-core-3.2.10.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/datanucleus-core-3.2.10.jar Dec 07, 2016 7:35:03 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:03 INFO Client: Uploading resource file:/hadoop/hdp/current/spark-client/lib/datanucleus-rdbms-3.2.9.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/datanucleus-rdbms-3.2.9.jar Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO Client: Uploading resource file:/hadoop/hdp/current/spark-client/lib/datanucleus-api-jdo-3.2.6.jar -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/datanucleus-api-jdo-3.2.6.jar Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO Client: Uploading resource file:/hadoop/hdp/current/spark-client/conf/hive-site.xml -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/hive-site.xml Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO Client: Uploading resource file:/tmp/spark-b1b81fe9-0cd5-4a89-ad8d-926abdca65f1/__spark_conf__4871478817518402722.zip -> hdfs://HDPDDMDEV/user/zeppelin-hdp_ddm_dev/.sparkStaging/application_1481021618182_0004/__spark_conf__4871478817518402722.zip Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO SecurityManager: Changing view acls to: livy,zeppelin-hdp_ddm_dev Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO SecurityManager: Changing modify acls to: livy,zeppelin-hdp_ddm_dev Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(livy, zeppelin-hdp_ddm_dev); users with modify permissions: Set(livy, zeppelin-hdp_ddm_dev) Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO Client: Submitting application 4 to ResourceManager Dec 07, 2016 7:35:04 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:04 INFO YarnClientImpl: Submitted application application_1481021618182_0004 Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:05 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:05 INFO Client: Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: client token: Token { kind: YARN_CLIENT_TOKEN, service: } Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: diagnostics: AM container is launched, waiting for AM container to Register with RM Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: ApplicationMaster host: N/A Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: ApplicationMaster RPC port: -1 Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: queue: default Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: start time: 1481096104540 Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: final status: UNDEFINED Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: tracking URL: http://rdeluh0001073.rz-c007-j650.basf-ag.de:8088/proxy/application_1481021618182_0004/ Dec 07, 2016 7:35:05 AM org.apache.spark.launcher.OutputRedirector redirect INFO: user: zeppelin-hdp_ddm_dev Dec 07, 2016 7:35:06 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:06 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:07 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:07 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:08 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:08 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:09 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:09 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:10 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:10 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:11 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:11 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:12 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:12 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) Dec 07, 2016 7:35:13 AM org.apache.spark.launcher.OutputRedirector redirect INFO: 16/12/07 07:35:13 INFO Client: Application report for application_1481021618182_0004 (state: ACCEPTED) 16/12/07 07:35:14 INFO RSCAppListener: Disconnect with app application_1481021618182_0004 16/12/07 07:35:47 INFO RSCClient: Received result for b3ae298b-9cc1-4df4-b149-9cafa45bb0e5 16/12/07 07:37:56 WARN RpcDispatcher: [ClientProtocol] Closing RPC channel with 1 outstanding RPCs.