How to configure zeppelin in Ambari 2.7.3 to use a yarn queue other than default?

[#1]

in Ambari 2.6.2.2, you can set Zeppelin to use a custom queue by going to Zepplen notebook --> Configs --> zeppelin_env_content and adding ZEPPELIN_JAVA_OPTS="-Dspark.yarn.queue=QUEUE_NAME_OTHER_THAN_DEFAULT" but it doesn't seem to work in Ambari 2.7.3. Zeppelin ends up using the default queue. How can I configure Zeppelin to use a custom queue other than the default in Ambari 2.7.3?

[#2]

@Rene Vargas

Add the following property in interpreter settings:

spark.yarn.queue=QUEUE_NAME (for spark interpreter)

[#3]

@scharan

Thank you for your response. Adding this property through Zeppelin seems to apply only to the current user in Zeppelin. Is there a way to set this property globally through Ambari?

[#4]

@Rene Vargas

It should work for all the users, you might have configured queue mapping in capacity scheduler.

you can also set in Advance Spark2-defaults from ambari at below peroperty

spark.yarn.queue=QUEUE NAME

[#5]

@Rene Vargas

Can you please try this to set the queue property globally through Ambari.

Ambari UI --> Zeppelin Notebook --> Configs --> Advanced --> "Advanced zeppelin-env" --> "zeppelin_env_content
"

Search for the following line:

export ZEPPELIN_JAVA_OPTS="-Dhdp.version={{full_stack_version}} -Dspark.executor.memory={{executor_mem}} -Dspark.executor.instances={{executor_instances}} -Dspark.yarn.queue={{spark_queue}}"

.

And then try to hardcode the value "-Dspark.yarn.queue={{spark_queue}}" with your desired queue name and then restart Zeppelin to see if it works for you.

Example:

export ZEPPELIN_JAVA_OPTS="-Dhdp.version={{full_stack_version}} -Dspark.executor.memory={{executor_mem}} -Dspark.executor.instances={{executor_instances}} -Dspark.yarn.queue=TestQueue"

.

Restart the Zeppelin.

.

NOTE: Please verify after restarting the Zeppelin that your following file is showing the updated Queue name or not?

# grep 'spark.yarn.queue'  /etc/zeppelin/conf/zeppelin-env.sh

.

[#6]

@Jay Kumar SenSharma

We have added "export ZEPPELIN_JAVA_OPTS='-Dspark.yarn.queue=zeppelin'" to the zeppelin configs in Ambari as we have done in the past with 2.6.2.2. We have since upgraded to 2.7.3 and this does not seem to be working in 2.7.3. New zeppelin users are using the "default" queue. I would like to know if that has changed in the new version and what can we do to fix it?


@scharan

Zeppelin by default uses spark's queue but when running, zeppelin uses up all available resources for the container it's in. We need to set a different queue for zeppelin so we can run it while still having available resources for separate spark queries.