Zeppelin Notebook will not start in Sandbox 2.4

[#1]

Zeppelin Notebook worked for a couple queries in the Sandbox Tutorial Lab #6. It stopped working after a Step 6.3.1 (Hive Query). When I attempt to restart Zeppelin Notebook I get the following error in the /var/lib/ambari-agent/data/errors-495.txt file:

urllib2.URLError: <urlopen error [Errno 111] Connection refused>

[#2]

@Michael Rife

On ambari home page what does it show in terms of resource usage on your sandbox? Are their any orphan jobs running?

[#3 · Accepted solution]

@Michael Rife, I don't know if you solved this, but I just had the issue and solved it for myself.

Reading the source code Zeppelin appears to be trying to connect to itself at that point, and since it hasn't started correctly it isn't able to start. I connected to the VM and confirm that nothing is listening on the Zeppelin port, so it's reasonable that it would fail to connect.

I look at the log for Zeppeloin and noticed multiple stack traces for an exception

javax.servlet.ServletException: Resource class org.apache.zeppelin.server.ZeppelinServer can not be instantiated due to InvocationTargetException

and went off Googling. It appears that this can be caused by an empty notebook file. So I looked for one and found it

[root@sandbox notebook]# find . -ls | more
2367730    4 drwxr-xr-x  17 zeppelin hadoop       4096 Apr 27 17:29 .
2371541    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2ANTDG878
2371542   16 -rw-r--r--   1 zeppelin hadoop      13609 Feb 26 20:47 ./2ANTDG878/note.json
2371561    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2BEQE47HR
2371562    4 -rw-r--r--   1 zeppelin hadoop       3546 Feb 26 20:47 ./2BEQE47HR/note.json
2371564    4 -rwxr-xr-x   1 zeppelin hadoop        326 Feb 26 20:47 ./update_all_notebooks.sh
2371557    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2BB5CUPUW
2371558 1048 -rw-r--r--   1 zeppelin hadoop    1071678 Feb 26 20:47 ./2BB5CUPUW/note.json
2371555    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2BAVUZ7NA
2371556  100 -rw-r--r--   1 zeppelin hadoop     102307 Feb 26 20:47 ./2BAVUZ7NA/note.json
2371539    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2ANT56EHN
2371540   16 -rw-r--r--   1 zeppelin hadoop      14906 Feb 26 20:47 ./2ANT56EHN/note.json
2371545    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2AS5TY6AQ
2371546   52 -rw-r--r--   1 zeppelin hadoop      52672 Feb 26 20:47 ./2AS5TY6AQ/note.json
2371537    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2A94M5J1Z
2371538   16 -rw-r--r--   1 zeppelin hadoop      13057 Feb 26 20:47 ./2A94M5J1Z/note.json
2371543    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2APFTN3NY
2371544 1292 -rw-r--r--   1 zeppelin hadoop    1319772 Feb 26 20:47 ./2APFTN3NY/note.json
2359541    4 drwxr-xr-x   2 zeppelin hadoop       4096 Apr 27 18:12 ./2BM4EHNF7
2359543    0 -rw-r--r--   1 zeppelin hadoop          0 Apr 27 18:12 ./2BM4EHNF7/note.json
2359427    4 drwxr-xr-x   2 zeppelin hadoop       4096 Apr 27 17:20 ./2BKDJ7EJQ
2359540   16 -rw-r--r--   1 zeppelin hadoop      16137 Apr 27 17:20 ./2BKDJ7EJQ/note.json
2371535  540 -rw-r--r--   1 zeppelin hadoop     549775 Mar 14 15:07 ./notebooks.zip
2371549    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2B48PF7SN
2371550   96 -rw-r--r--   1 zeppelin hadoop      96143 Feb 26 20:47 ./2B48PF7SN/note.json
2371559    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2BBBW75VS
2371560 1188 -rw-r--r--   1 zeppelin hadoop    1213695 Feb 26 20:47 ./2BBBW75VS/note.json
2371553    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2B522V3X8
2371554  268 -rw-r--r--   1 zeppelin hadoop     274336 Feb 26 20:47 ./2B522V3X8/note.json
2371563    8 -rw-r--r--   1 zeppelin hadoop       6433 Feb 26 20:47 ./README.md
2371547    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2B21B3AYC
2371548   76 -rw-r--r--   1 zeppelin hadoop      77501 Feb 26 20:47 ./2B21B3AYC/note.json
2371551    4 drwxr-xr-x   2 zeppelin hadoop       4096 Feb 26 20:47 ./2B4TWGC8M
2371552   48 -rw-r--r--   1 zeppelin hadoop      46910 Feb 26 20:47 ./2B4TWGC8M/note.json

I moved the directory with the offending file (./2BM4EHNF7) off to /tmp and it worked.

A quick way to do this is to SSH into the sandbox and use a one liner to move the files out:

$  ssh root@127.0.0.1 -p 2222
root@127.0.0.1's password: 
Last login: Wed Apr 27 18:52:40 2016 from 10.0.2.2
[root@sandbox ~]# BAD_FILE=`find /usr/hdp/current/zeppelin-server/lib/notebook -name "*.json" -size 0` ; mv `dirname $BAD_FILE` /tmp/

[#4]

@Chuck D'Antonio, I had the same problem. I had an empty notebook (empty note.json file). I moved the directory to /tmp and the Zeppelin Notebook server started. I assume that I can now drop the directory I moved to /tmp.

Thanks!!

[#5]

Thank you so much for your answer I had the same problem this morning and I thought it was a proxy problem but actually I had an empty note in Zeppelin. I hope this tiny problem will be fixed in the next versions of Zeppelin Notebook.

When I check the status of zeppelin-daemon.sh it tells me "Zeppelin running but process is dead" can anyone knows the reason for this error ?