Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Zeppelin and Java 8 on HDP 2.5

avatar
Expert Contributor

We could install Zeppelin through Ambari, and it runs with green light. However, when I open and try %sql (I guess Spark interpereter), it got error. When I check the zeppelin log file, I see the out of memory error with -XX:MaxPermSize=512m. I know this parameter is untill jdk 7. The jdk 8 has a different new parameter. I'm certain we need to use ZEPPELIN_MEM to set the new parameter, but I don't know how to do it correctly through Ambari.

Could anyone help on this?

1 ACCEPTED SOLUTION

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
6 REPLIES 6

avatar
Master Mentor

@Shigeru Takehara

There are various kind of OutOfMemory errors. (Example: OutOfMemory in Native Space, OurOfMemory in Heap Space, OutOfMemory due to GC overhead....etc)

As you said "I see the out of memory error with -XX:MaxPermSize=512m", Can you please let us know which kind of OutOfMemory error are you getting? Sharing log will be more useful.

Even if you are using Java8 and still if the "-XX:MaxPermSize=512m" is applied then java8 will simply ignore it.

.

If you want to make any changes in the Zeppelin memory settings then you can login to ambari UI and the navigate to

"Zeppelin Notebook" --> "Configs" (Tab) --> "Advanced zeppelin-env"

'and then find the "zeppelin_env_content" there you will find "export ZEPPELIN_MEM" that you can edit.

avatar
Expert Contributor

log file says:

INFO [2017-04-07 11:53:58,682] ({pool-2-thread-4} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1491580438681 started by scheduler org.apache.zeppelin.spark.SparkInterpreter283573820 INFO [2017-04-07 11:54:02,145] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Changing view acls to: zeppelin INFO [2017-04-07 11:54:02,145] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Changing modify acls to: zeppelin INFO [2017-04-07 11:54:02,145] ({pool-2-thread-4} Logging.scala[logInfo]:58) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(zeppelin); users with modify permissions: Set(zeppelin) INFO [2017-04-07 11:54:05,322] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Starting HTTP Server INFO [2017-04-07 11:54:05,945] ({pool-2-thread-4} Server.java[doStart]:272) - jetty-8.y.z-SNAPSHOT INFO [2017-04-07 11:54:08,337] ({pool-2-thread-4} AbstractConnector.java[doStart]:338) - Started SocketConnector@0.0.0.0:22900 INFO [2017-04-07 11:54:08,338] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Successfully started service 'HTTP class server' on port 22900. ERROR [2017-04-07 11:54:10,658] ({pool-2-thread-4} Job.java[run]:189) - Job failed java.lang.OutOfMemoryError: GC overhead limit exceeded INFO [2017-04-07 11:54:10,658] ({pool-2-thread-4} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1491580438681 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter283573820 INFO [2017-04-07 11:54:11,954] ({pool-1-thread-18} Logging.scala[logInfo]:58) - Changing view acls to: zeppelin INFO [2017-04-07 11:54:11,954] ({pool-1-thread-18} Logging.scala[logInfo]:58) - Changing modify acls to: zeppelin INFO [2017-04-07 11:54:11,955] ({pool-1-thread-18} Logging.scala[logInfo]:58) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(zeppelin); users with modify permissions: Set(zeppelin)

"xxx.out" file says:

Exception in thread "qtp686649452-427" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1855) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2068) at org.spark-project.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342) at org.spark-project.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526) at org.spark-project.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44) at org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) Exception in thread "pool-1-thread-4" Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "pool-1-thread-4" Exception in thread "pool-1-thread-7" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-6" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-9" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1834748429-966" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-8" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-13" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-12" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-11" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-1" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-3" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp184565796-1023" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1834748429-962" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp851486649-779" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1459091505-28" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1613009598-14753" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-18" java.lang.OutOfMemoryError: GC overhead limit exceeded

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Expert Contributor

Yes, that's right. On Ambari, should we add zeppelin_mem, let's say -XmXXXXm to Custom zeppelin-env?

avatar
Master Mentor

@Shigeru Takehara

If you want to make any changes in the Zeppelin memory settings then you can login to ambari UI and the navigate to

"Zeppelin Notebook" --> "Configs" (Tab) --> "Advanced zeppelin-env"

'and then find the "zeppelin_env_content" there you will find "export ZEPPELIN_MEM" that you can edit.

avatar
Expert Contributor

I added:

export ZEPPELIN_MEM="-Xms1024m -Xmx2048m -XX:MaxMetaspaceSize=512m"

The out of memory issue was fixed; however,

I got another new issue related to yarn configuration. I will ask it with "new question".

Thanks!