I need to map YARN queues to analytic phoenix queries. That is, when
user1 runs "select col1, col2 from table1" user1 runs it in yarn-queue1 and that way we can make sure adequate resources is given to the queue to avoid contention with lower priority jobs.
However, I have tested with the
parameters but i am not able to see the queries that are executed in YARN.
How do i make sure these queries are catered for?
N.B: We use squirrel to connect to Phoenix.
Unfortunately, Phoenix (HBase) is not yet integrated in YARN by default (HDP 2.6.5 or lower, 3.x I don't know). So the ressources of HBase loads are managed outside of YARN. That's the reason why it is recommended that the YARN Property yarn.nodemanager.resource.memory-mb should not set to all your cluster ressources. There should be enough space for HBase workloads and things like the Operating System. However, there are some workarounds to manage Phoenix/HBase Ressources.