Member since
04-01-2022
7
Posts
0
Kudos Received
0
Solutions
02-27-2023
01:45 AM
Hello @Scout First, we need to understand we tried to enable tag-based policy only for Hive jobs because those jobs submitted by proxy user hive. After we enable the tab-bases policy the job submitted with hive user instead of proxy user Instructions Step 1: Enable application tag placement and whitelist hive In Cloudera Manager, select the YARN service. Click the Configuration tab. Search for ResourceManager. In the Filters pane, under Scope, select ResourceManager. In ResourceManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml add the following. Set these two yarn configurations: yarn.resourcemanager.application-tag-based-placement.enable=true; yarn.resourcemanager.application-tag-based-placement.username.whitelist=hive; Step 2: You need to check, and if not found, add several configuration properties to allow placement of the Hive workload on the Yarn queue manager. In Cloudera Manager, click Clusters > Hive on Tez> Configuration. Search for the Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml setting. In the Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml setting, click +. In Name enter the property hive.server2.tez.initialize.default.sessions and in value enter false. In Name enter the property hive.server2.tez.queue.access.check and in value enter true. In Name enter the property hive.server2.tez.sessions.custom.queue.allowed and in value enter true. Next you need to add placement rules as below: Add placement rules >> This rule matches = hive_user >> Custom policy = <yarn.queue.name> (where you want to submit the job. Note: Order of policy also played vital role, please order properly to move the job in aligned queue.
... View more
02-06-2023
07:50 AM
@Kirankathe Could you please share error message reflecting application log and diagnostic message as well. Click on Accepted state application_id you will be able see diagnostic message
... View more