Zeppelin service is running (I can use Zeppelin view) but Ambari reports it is down. Some investigation revealed this is probably due to the fact that when Zeppelin was starting it was not able to create a PID file. The exception was thrown
mkdir: cannot create directory ‘/var/run/zeppelin’: Permission denied
If I manually give all permissions to /var/run/ folder it starts to work properly.
It looks like I am not only one facing this issue as I saw other users already reported it. It is still not clear however what is a workaround for the problem? Giving permissions to /var/run/ folder only works as one time solution. As this folder is recreated on reboot, once I restart computer the problem starts over again...
That is known OS issue.
What i did to circumvent this was:
On ambari server host. under /var/lib/ambari-server/resources/scripts you will find a script called configs.sh,
Change the value of the zeppelin PID directory as below:
STOP zeppelin Service.
/configs.sh -u $ambari_user -p $password get localhost $cluster_name zeppelin-env
id -a zeppelin
/configs.sh -u $ambari_user -p $password set localhost $cluster_name zeppelin-env "zeppelin_pid_dir" "/var/run/user/<zeppelin ID>/zeppelin"
mkdir -p /var/run/user/<zeppelin ID>/zeppelin
chown -R zeppelin:zeppelin /var/run/user
and then start the zeppelin service from ambari.
You should now see the new PID location under advanced zeppelin env
Please restart the server after this and see if the value holds.
I however was not sure what "id -a zeppelin" means in your post? My best guess was that you suggested to find some zeppelin id. But how do I find it? The /configs.sh -u $ambari_user -p $password get localhost $cluster_name zeppelin-env command returned a pretty big output. And I was not sure where I look for zeppelin id?
it looks related to an already reported issue:
Please use the workaround to manually assign the permission to the mentioned directory, until the fix.
mkdir -p /var/run/zeppelin chown -R zeppelin:zeppelin /var/run/zeppelin
@Dmitry Otbleskcould you please check where your /var/run directory is mounted on tmpfs using
df -k /var/run
if it is on tmpfs that could cause this dir to remove once you reboot the node
if not then there is no harm of creating /var/run/zeppelin folder manually and give it permission as zeppelin:hadoop
For properties that are disabled like that, you have to use the helper script configs.sh like so:
./configs.sh set localhost c1 zeppelin-env zeppelin_pid_dir /var/run/zeppelin/piddir
(Replace variables as required for hostname, cluster name, pid dir etc.)
After you set it, you may need to reload the config page to see the change and restart your services.