Member since
03-20-2023
1
Post
0
Kudos Received
0
Solutions
03-20-2023
03:21 AM
Hi, I've tried to connect hadoop, spark, and Zeppelin, and I confirmed that hadoop and spark server is working well. But I can't connect to Zeppelin server. This is a log of zeppelin: WARN [2023-03-20 17:43:32,397] ({main} ZeppelinConfiguration.java[<init>]:85) - Failed to load XML configuration, proceeding with a default,for a stacktrace activate the debug log INFO [2023-03-20 17:43:32,421] ({main} ZeppelinConfiguration.java[create]:135) - Server Host: 127.0.0.1 INFO [2023-03-20 17:43:32,421] ({main} ZeppelinConfiguration.java[create]:139) - Server Port: 8080 INFO [2023-03-20 17:43:32,421] ({main} ZeppelinConfiguration.java[create]:141) - Context Path: / INFO [2023-03-20 17:43:32,424] ({main} ZeppelinConfiguration.java[create]:142) - Zeppelin Version: 0.10.1 INFO [2023-03-20 17:43:32,448] ({main} Log.java[initialized]:169) - Logging initialized @1905ms to org.eclipse.jetty.util.log.Slf4jLog WARN [2023-03-20 17:43:32,997] ({main} ZeppelinConfiguration.java[getConfigFSDir]:653) - zeppelin.config.fs.dir is not specified, fall back to local conf directory zeppelin.conf.dir WARN [2023-03-20 17:43:33,005] ({main} ZeppelinConfiguration.java[getConfigFSDir]:653) - zeppelin.config.fs.dir is not specified, fall back to local conf directory zeppelin.conf.dir WARN [2023-03-20 17:43:33,005] ({main} ZeppelinConfiguration.java[getConfigFSDir]:653) - zeppelin.config.fs.dir is not specified, fall back to local conf directory zeppelin.conf.dir WARN [2023-03-20 17:43:33,067] ({main} LocalConfigStorage.java[loadCredentials]:88) - Credential file /home/nahyeon/apps/zeppelin/conf/credentials.json is not existed INFO [2023-03-20 17:43:33,218] ({ImmediateThread-1679301812982} PluginManager.java[loadNotebookRepo]:78) - Loading NotebookRepo Plugin: org.apache.zeppelin.notebook.repo.GitNotebookRepo INFO [2023-03-20 17:43:33,534] ({ImmediateThread-1679301812982} VFSNotebookRepo.java[setNotebookDirectory]:69) - Using notebookDir: /home/nahyeon/apps/zeppelin/notebook INFO [2023-03-20 17:43:33,718] ({main} ZeppelinServer.java[setupWebAppContext]:577) - warPath is: /home/nahyeon/apps/zeppelin/zeppelin-web-0.10.1.war INFO [2023-03-20 17:43:33,719] ({main} ZeppelinServer.java[setupWebAppContext]:590) - ZeppelinServer Webapp path: /home/nahyeon/apps/zeppelin/webapps INFO [2023-03-20 17:43:33,802] ({main} ZeppelinServer.java[setupWebAppContext]:577) - warPath is: /home/nahyeon/apps/zeppelin/zeppelin-web-angular-0.10.1.war INFO [2023-03-20 17:43:33,803] ({main} ZeppelinServer.java[setupWebAppContext]:590) - ZeppelinServer Webapp path: /home/nahyeon/apps/zeppelin/webapps/next INFO [2023-03-20 17:43:33,820] ({ImmediateThread-1679301812982} GitNotebookRepo.java[init]:76) - Opening a git repo at '/home/nahyeon/apps/zeppelin/notebook' INFO [2023-03-20 17:43:33,911] ({main} NotebookServer.java[<init>]:154) - NotebookServer instantiated: org.apache.zeppelin.socket.NotebookServer@1500b2f3 INFO [2023-03-20 17:43:33,911] ({main} NotebookServer.java[setNotebook]:165) - Injected NotebookProvider INFO [2023-03-20 17:43:33,911] ({main} NotebookServer.java[setServiceLocator]:159) - Injected ServiceLocator: ServiceLocatorImpl(shared-locator,0,787738361) INFO [2023-03-20 17:43:33,912] ({main} NotebookServer.java[setNotebookService]:172) - Injected NotebookServiceProvider INFO [2023-03-20 17:43:33,912] ({main} NotebookServer.java[setAuthorizationServiceProvider]:179) - Injected NotebookAuthorizationServiceProvider INFO [2023-03-20 17:43:33,912] ({main} NotebookServer.java[setConnectionManagerProvider]:185) - Injected ConnectionManagerProvider INFO [2023-03-20 17:43:33,914] ({main} ZeppelinServer.java[setupClusterManagerServer]:467) - Cluster mode is disabled INFO [2023-03-20 17:43:33,914] ({main} ZeppelinServer.java[main]:251) - Starting zeppelin server INFO [2023-03-20 17:43:33,917] ({main} Server.java[doStart]:360) - jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 1.8.0_362-8u362-ga-0ubuntu1~22.04-b09 INFO [2023-03-20 17:43:34,063] ({main} StandardDescriptorProcessor.java[visitServlet]:276) - NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet INFO [2023-03-20 17:43:34,075] ({main} DefaultSessionIdManager.java[doStart]:334) - DefaultSessionIdManager workerName=node0 INFO [2023-03-20 17:43:34,075] ({main} DefaultSessionIdManager.java[doStart]:339) - No SessionScavenger set, using defaults INFO [2023-03-20 17:43:34,076] ({main} HouseKeeper.java[startScavenging]:140) - node0 Scavenging every 600000ms INFO [2023-03-20 17:43:34,089] ({main} ContextHandler.java[log]:2303) - Initializing Shiro environment INFO [2023-03-20 17:43:34,089] ({main} EnvironmentLoader.java[initEnvironment]:133) - Starting Shiro environment initialization. INFO [2023-03-20 17:43:34,422] ({main} EnvironmentLoader.java[initEnvironment]:147) - Shiro environment initialized in 333 ms. INFO [2023-03-20 17:43:37,854] ({main} ContextHandler.java[doStart]:860) - Started o.e.j.w.WebAppContext@563f38c4{zeppelin-web,/,jar:file:///home/nahyeon/apps/zeppelin-0.10.1-bin-all/zeppelin-web-0.10.1.war!/,AVAILABLE}{/home/nahyeon/apps/zeppelin/zeppelin-web-0.10.1.war} INFO [2023-03-20 17:43:37,897] ({main} StandardDescriptorProcessor.java[visitServlet]:276) - NO JSP Support for /next, did not find org.eclipse.jetty.jsp.JettyJspServlet INFO [2023-03-20 17:43:37,902] ({main} ContextHandler.java[log]:2303) - Initializing Shiro environment INFO [2023-03-20 17:43:37,902] ({main} EnvironmentLoader.java[initEnvironment]:133) - Starting Shiro environment initialization. INFO [2023-03-20 17:43:37,921] ({main} EnvironmentLoader.java[initEnvironment]:147) - Shiro environment initialized in 19 ms. INFO [2023-03-20 17:43:38,604] ({main} ContextHandler.java[doStart]:860) - Started o.e.j.w.WebAppContext@53045c6c{zeppelin-web-angular,/next,jar:file:///home/nahyeon/apps/zeppelin-0.10.1-bin-all/zeppelin-web-angular-0.10.1.war!/,AVAILABLE}{/home/nahyeon/apps/zeppelin/zeppelin-web-angular-0.10.1.war} ERROR [2023-03-20 17:43:38,633] ({main} ZeppelinServer.java[main]:262) - Error while running jettyServer java.io.IOException: Failed to bind to /127.0.0.1:8080 at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349) at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.server.Server.doStart(Server.java:386) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:253) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:461) at sun.nio.ch.Net.bind(Net.java:453) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85) at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:345) ... 7 more --------------------------------- And this is the code I changed in zeppelin-site.xml.template(in default, it didn't have zeppelin-site.xml) <property> <name>zeppelin.server.addr</name> <value>127.0.0.1</value> <description>Server binding address</description> </property> <property> <name>zeppelin.server.port</name> <value>9999</value> <description>Server port.</description> </property ------------------------------------------ And this is the code I inserted in zeppelin-env.sh.template(also, no zeppelin-env.sh) export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 export MASTER=spark://localhost:7077 export ZEPPELIN_MEM="-Xms1g -Xmx8g -XX:MaxPermSize=1g" export ZEPPELIN_INTP_MEM="-Xms1g -Xmx8g -XX:MaxPermSize=1g" export SPARK_HOME=/home/nahyeon/apps/spark export HADOOP_CONF_DIR=/home/nahyeon/hdoop/hadoop-3.3.4/etc/hadoop export PYSPARK_PYTHON=python3 export PYTHONPATH=python3 export ZEPPELIN_PORT=9999 ----------------------------------- **paths of ~/.bashrc are: export ZEPPELIN_HOME=/home/nahyeon/apps/zeppelin export PATH=$PATH:$ZEPPELIN_HOME/bin It seems like although I chose the port number which is different from spark's default port number(8080), it doesn't apply. And maybe it's because it couldn't find XML configuration, according to the first line of the log above. But I'm not sure I understood well, and I have no idea how to fix it. Could anybody help me?
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache Zeppelin
-
HDFS