Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

zeppelin + Livy failes

avatar
Expert Contributor

No idea why but livy stopped working today.

It spits the following out non stop. No idea why anyone got an idea of where to look.

org.apache.zeppelin.livy.LivyException: Session 349 is finished, appId: application_1522011783190_0196, log: [	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866), 	at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:66), 	at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:766), 	at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala), 18/03/29 17:06:54 INFO ApplicationMaster: Final app status: FAILED, exitCode: 10, (reason: Uncaught exception: java.lang.ClassNotFoundException: org.apache.livy.rsc.driver.RSCDriverBootstrapper), 18/03/29 17:06:54 INFO ApplicationMaster: Unregistering ApplicationMaster with FAILED (diag message: Uncaught exception: java.lang.ClassNotFoundException: org.apache.livy.rsc.driver.RSCDriverBootstrapper), 18/03/29 17:06:54 INFO ApplicationMaster: Deleting staging directory hdfs://lrdccdhm01.cloud.res.bngf.local:8020/user/andm013/.sparkStaging/application_1522011783190_0196, 18/03/29 17:06:55 INFO ShutdownHookManager: Shutdown hook called, , Failing this attempt. Failing the application.]
Livy logs:
18/03/29 17:07:35 INFO RSCClient: Failing pending job 94534a77-1855-45c9-bc18-8e53c8e42165 due to shutdown.
18/03/29 17:07:35 INFO InteractiveSession: Failed to ping RSC driver for session 348. Killing application.
18/03/29 17:07:35 INFO InteractiveSession: Stopping InteractiveSession 348...
18/03/29 17:07:35 WARN InteractiveSession: (Fail to get rsc uri,java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException: Timed out waiting for context to start.)
18/03/29 17:07:38 INFO InteractiveSession: Stopped InteractiveSession 348.
18/03/29 17:08:09 ERROR RSCCli
ent: Failed to connect to context.
java.util.concurrent.TimeoutException: Timed out waiting for context to start.
        at org.apache.livy.rsc.ContextLauncher.connectTimeout(ContextLauncher.java:134)
        at org.apache.livy.rsc.ContextLauncher.access$300(ContextLauncher.java:63)
        at org.apache.livy.rsc.ContextLauncher$2.run(ContextLauncher.java:122)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
        at java.lang.Thread.run(Thread.java:745)
18/03/29 17:08:09 INFO RSCClient: Failing pending job 04982890-5fc8-4015-98a9-ad888b4e6b45 due to shutdown.
18/03/29 17:08:09 INFO InteractiveSession: Failed to ping RSC driver for session 349. Killing application.
18/03/29 17:08:09 INFO InteractiveSession: Stopping InteractiveSession 349...
18/03/29 17:08:09 WARN InteractiveSession: (Fail to get rsc uri,java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException: Timed out waiting for context to start.)
18/03/29 17:08:09 INFO InteractiveSession: Stopped InteractiveSession 349.


1 REPLY 1

avatar
Expert Contributor

Turns out I had added the the LLAP pibs twice.