Created 01-26-2023 09:01 AM
Hi, trying to set placement rules as described in https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/yarn-allocate-resources/topics/yarn-placement-rule-policies.html, but tasks still in default queue.
Also tried to enable tag-based authorization with the same result.
What could be wrong?
Created 02-26-2023 05:32 AM
Please share a screenshot of your placement rules
Created 02-27-2023 01:45 AM
Hello @Scout
First, we need to understand we tried to enable tag-based policy only for Hive jobs because those jobs submitted by proxy user hive. After we enable the tab-bases policy the job submitted with hive user instead of proxy user
Step 1:
Enable application tag placement and whitelist hive
Step 2:
You need to check, and if not found, add several configuration properties to allow placement of the Hive workload on the Yarn queue manager.
Next you need to add placement rules as below:
Add placement rules >> This rule matches = hive_user >> Custom policy = <yarn.queue.name> (where you want to submit the job.
Note: Order of policy also played vital role, please order properly to move the job in aligned queue.
Created 03-24-2023 08:22 AM
Hello, sorry for a big delay.
That solution didn't worked for me.
I've set all parameters in Yarn and Hive-on-Tez, but test user going to default query anyway.
Maybe I can check logs or something for that placement rules?
Created 03-02-2023 07:05 AM
@Scout, Has any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
Regards,
Vidya Sargur,Created 03-29-2023 02:29 AM
Nope.
We can set queue for spark with --queue argument and it would work.
But the rest requests are ignoring yarn placement rules.
Created 04-01-2023 08:15 AM
Please share a specific use case, ie. Pick a sample user job that is expected to go to a specific queue and share its placement rules screenshot
Created 04-05-2023 04:00 AM
Tried to run spark and zeppelin jobs.
application placement rules are:
-submitting user's name = <user>
-place the application into queue named for user / set default queue to / use custom placement rules
- tried to use static and dynamic queues
The results are the same: application runs in default queue
Created 04-01-2023 08:30 AM
Hi, I am not able to open a case but I work like a agent for cloudera , I was running the stock market and I am not able to withdraw my deposit since the system give me high package and I can’t manage to cover the overdraft again.. what can I do?
Created 04-01-2023 08:35 AM
someone who can help me with this, I don’t want to lose my money?