if yarn.scheduler.capacity.queue-mappings-override.enable is set to false then user's have more control on which queue they want to submit the application either by setting the queue name or from default mapping, in this case user specified queue name takes the precedence , but if yarn.scheduler.capacity.queue-mappings-override.enable set to true then admin's have more control and admin can override user's explicit settings. ... View more

@Sushant ,There are mainly two ways to enable Dynamic Resource Allocation. 1) Spark level Enable DRA in spark. Follow below setup instruction to enable DRA. This way all spark application will run with DRA enabled. https:///content/supportkb/49510/how-to-enable-dynamic-resource-allocation-in-spark.html 2) Zeppelin level You can choose to enable DRA in Zeppelin interpreter. You can use Livy interpreter to run spark-shell or pyspark jobs. https://zeppelin.apache.org/docs/0.6.1/interpreter/livy.html ... View more

Hi @Sushant Glad that worked for you, in case can you please accept that answer. in response to control the resource monitoring, not that I am aware of, but I believe you may 'not' need to prevent user 1 to see the application submitted by user1 or other user. as this does not contain any data (unless explisitly prints out to STDOUT). on the other hand you can manage the access with (authorization)SPNEGO for web UI. ... View more

@jack jack First you should open a new HCC thread that way each individual queries are tracked separately. ... View more