Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Ambari doesn't install llap

Ambari doesn't install llap

Expert Contributor

capture4.png

Hi, help me please to understand why Ambari doesn't install llap.
Yarn Pre-emption is turn on.

When i try enabling LLAP, i am getting error in ambari-server.log

Installation Wizard doesn't show list hosts for LLAP.

60417-capture4.png

<a href="/storage/attachments/60415-log.txt">log.txt</a>06 Feb 2018 11:44:13,929  INFO [ambari-client-thread-75953] StackAdvisorRunner:47 - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, actionDirectory=/var/run/ambari-server/stack-recommendations/37, command=recommend-configuration-dependencies06 Feb 2018 11:44:13,937  INFO [ambari-client-thread-75953] StackAdvisorRunner:61 - Stack-advisor output=/var/run/ambari-server/stack-recommendations/37/stackadvisor.out, error=/var/run/ambari-server/stack-recommendations/37/stackadvisor.err06 Feb 2018 11:44:14,819  INFO [ambari-client-thread-75953] StackAdvisorRunner:69 - Stack advisor output files06 Feb 2018 11:44:14,820  INFO [ambari-client-thread-75953] StackAdvisorRunner:70 -  advisor script stdout: StackAdvisor implementation for stack HDP, version 2.0.6 was loadedStackAdvisor implementation for stack HDP, version 2.1 was loadedStackAdvisor implementation for stack HDP, version 2.2 was loadedStackAdvisor implementation for stack HDP, version 2.3 was loadedStackAdvisor implementation for stack HDP, version 2.4 was loadedStackAdvisor implementation for stack HDP, version 2.5 was loadedStackAdvisor implementation for stack HDP, version 2.6 was loadedReturning HDP26StackAdvisor implementation2018-02-06 11:44:14,707 - Memory for YARN apps - cluster[totalAvailableRam]: 174082018-02-06 11:44:14,707 - call type context : {'call_type': 'recommendConfigurationDependencies'}2018-02-06 11:44:14,707 - user operation context : EditConfig2018-02-06 11:44:14,707 - Full context: callContext = recommendConfigurationDependencies and operation = EditConfig and adding YARN = False and old value exists = None2018-02-06 11:44:14,708 - Using user provided yarn.scheduler.minimum-allocation-mb = 20482018-02-06 11:44:14,708 - Minimum ram per container due to user input - cluster[yarnMinContainerSize]: 20482018-02-06 11:44:14,708 - Containers per node - cluster[containers]: 82018-02-06 11:44:14,708 - Ram per containers before normalization - cluster[ramPerContainer]: 21762018-02-06 11:44:14,708 - Min container size - cluster[yarnMinContainerSize]: 20482018-02-06 11:44:14,708 - Available memory for map - cluster[mapMemory]: 20482018-02-06 11:44:14,708 - Available memory for reduce - cluster[reduceMemory]: 20482018-02-06 11:44:14,708 - Available memory for am - cluster[amMemory]: 2048ServiceAdvisor implementation for service NIFI was loadedFailed to load or create ServiceAdvisor implementation for service REGISTRY: Expecting class name REGISTRY030ServiceAdvisor but it was not found.Failed to load or create ServiceAdvisor implementation for service STREAMLINE: Expecting class name STREAMLINE050ServiceAdvisor but it was not found.max_inmemory_regions: -2.9Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/AMBARI_METRICS.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HBASE.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HDFS.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HIVE.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HOST.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/STORM.txtProcessing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/YARN.txtmetrics length: 3332018-02-06 11:44:14,717 - Constructing atlas.rest.address=http://*.*.*:210002018-02-06 11:44:14,720 - Setting Hbase Repo user for Ranger.2018-02-06 11:44:14,721 - Hive Server and Hive Server Interactive (if enabled) Host List : set([u'*.*.*'])2018-02-06 11:44:14,721 - Updated hadoop.proxyuser.hive.hosts as : *.*.*2018-02-06 11:44:14,721 - Updated hadoop.proxyuser.hive.hosts as : *2018-02-06 11:44:14,722 - Updated hadoop.proxyuser.hive.hosts as : *2018-02-06 11:44:14,722 - Updated hadoop.proxyuser.hive.hosts as : *.*.*2018-02-06 11:44:14,722 - Updated hadoop.proxyuser.hive.hosts as : *2018-02-06 11:44:14,723 - Setting HDFS Repo user for Ranger.2018-02-06 11:44:14,723 - DBG: Invoked recommendHiveConfiguration2018-02-06 11:44:14,724 - call type context : {'call_type': 'recommendConfigurationDependencies'}2018-02-06 11:44:14,724 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,725 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,725 - DBG: Entered updateLlapConfigs2018-02-06 11:44:14,725 - Is cluster create operation ? = False2018-02-06 11:44:14,725 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.2018-02-06 11:44:14,726 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.2018-02-06 11:44:14,726 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,726 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,726 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.2018-02-06 11:44:14,726 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'hunger_games', 'label': 'hunger_games'}, {'value': 'load_raw', 'label': 'load_raw'}, {'value': 'low_prior', 'label': 'low_prior'}, {'value': 'queries', 'label': 'queries'}]2018-02-06 11:44:14,726 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,727 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,727 - YARN leaf Queues = set(['default', 'hunger_games', 'low_prior', 'load_raw', 'queries'])2018-02-06 11:44:14,727 - DBG: llap_named_queue_selected_in_curr_invocation = False2018-02-06 11:44:14,727 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'True'.2018-02-06 11:44:14,727 - Selected YARN queue for LLAP is : 'default'. Current YARN queues : ['default', 'hunger_games', 'low_prior', 'load_raw', 'queries']. Setting 'Number of LLAP nodes' visibility to 'False'.2018-02-06 11:44:14,727 - DBG: Performing LLAP config calculations ......2018-02-06 11:44:14,727 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.02018-02-06 11:44:14,727 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 20482018-02-06 11:44:14,727 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 10242018-02-06 11:44:14,728 - DBG Calculated normalized_tez_am_container_size : 2048.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.02018-02-06 11:44:14,728 - DBG: Calculated and returning 'hive_tez_container_size' : 30722018-02-06 11:44:14,728 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.02018-02-06 11:44:14,728 - DBG: Calculated 'slider_am_container_size' : 2048.0, using following: yarn_min_container_size : 2048.02018-02-06 11:44:14,728 - DBG: Calculated 'min_memory_required': 8192.0 using following : slider_am_container_size: 2048.0, normalized_tez_am_container_size : 2048.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 2048.02018-02-06 11:44:14,728 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 8192.0, yarn_nm_mem_in_mb_normalized : 24576.02018-02-06 11:44:14,728 - DBG: selected_queue_is_ambari_managed_llap = False2018-02-06 11:44:14,728 - Entered __getSelectedQueueTotalCap fn() with llap_daemon_selected_queue_name= 'default'.2018-02-06 11:44:14,728 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity2018-02-06 11:44:14,729 - Queue list : ['root', 'default']2018-02-06 11:44:14,729 - DBG: Selected queue name as: yarn.scheduler.capacity.root.capacity2018-02-06 11:44:14,729 - Total capacity available for queue root is : 49152.02018-02-06 11:44:14,729 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity2018-02-06 11:44:14,729 - Total capacity available for queue default is : 4915.22018-02-06 11:44:14,729 - DBG: Calculated 'default' queue available capacity : 4096.0, using following: llap_daemon_selected_queue_cap : 4915.2, yarn_min_container_size : 2048.02018-02-06 11:44:14,729 - DBG: Calculated 'num_llap_nodes_requested' : 1.0, using following: total_llap_mem_normalized : 4096.0, yarn_nm_mem_in_mb_normalized : 24576.02018-02-06 11:44:14,729 - Setting config 'num_llap_nodes' as : 1.02018-02-06 11:44:14,729 - Returning default AM percent value : '0.1' for queue : default2018-02-06 11:44:14,730 - DBG: Calculated 'hive_tez_am_cap_available' : 409.6, using following: queue_am_fraction_perc : 0.1, total_llap_mem_normalized : 4096.02018-02-06 11:44:14,730 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 2048.0, using following : total_llap_mem_normalized : 4096.0, slider_am_container_size : 2048.02018-02-06 11:44:14,730 - Not enough capacity available on the cluster to run LLAP2018-02-06 11:44:14,730 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration2018-02-06 11:44:14,730 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 20482018-02-06 11:44:14,730 - Setting Yarn Repo user for Ranger.2018-02-06 11:44:14,731 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,731 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,732 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'True'.2018-02-06 11:44:14,732 - Setting Hive Repo user for Ranger.2018-02-06 11:44:14,733 - Not setting Kafka Repo user for Ranger.2018-02-06 11:44:14,733 - call type context : {'call_type': 'recommendConfigurationDependencies'}2018-02-06 11:44:14,733 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,733 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,734 - DBG: Entered updateLlapConfigs2018-02-06 11:44:14,734 - Is cluster create operation ? = False2018-02-06 11:44:14,734 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.2018-02-06 11:44:14,734 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.2018-02-06 11:44:14,735 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,735 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,735 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.2018-02-06 11:44:14,735 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'hunger_games', 'label': 'hunger_games'}, {'value': 'load_raw', 'label': 'load_raw'}, {'value': 'low_prior', 'label': 'low_prior'}, {'value': 'queries', 'label': 'queries'}]2018-02-06 11:44:14,735 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,735 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,735 - YARN leaf Queues = set(['default', 'hunger_games', 'low_prior', 'load_raw', 'queries'])2018-02-06 11:44:14,736 - DBG: llap_named_queue_selected_in_curr_invocation = False2018-02-06 11:44:14,736 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'True'.2018-02-06 11:44:14,736 - Selected YARN queue for LLAP is : 'default'. Current YARN queues : ['default', 'hunger_games', 'low_prior', 'load_raw', 'queries']. Setting 'Number of LLAP nodes' visibility to 'False'.2018-02-06 11:44:14,736 - DBG: Performing LLAP config calculations ......2018-02-06 11:44:14,736 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.02018-02-06 11:44:14,736 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 20482018-02-06 11:44:14,736 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 10242018-02-06 11:44:14,736 - DBG Calculated normalized_tez_am_container_size : 2048.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.02018-02-06 11:44:14,736 - DBG: Calculated and returning 'hive_tez_container_size' : 30722018-02-06 11:44:14,737 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.02018-02-06 11:44:14,737 - DBG: Calculated 'slider_am_container_size' : 2048.0, using following: yarn_min_container_size : 2048.02018-02-06 11:44:14,737 - DBG: Calculated 'min_memory_required': 8192.0 using following : slider_am_container_size: 2048.0, normalized_tez_am_container_size : 2048.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 2048.02018-02-06 11:44:14,737 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 8192.0, yarn_nm_mem_in_mb_normalized : 24576.02018-02-06 11:44:14,737 - DBG: selected_queue_is_ambari_managed_llap = False2018-02-06 11:44:14,737 - Entered __getSelectedQueueTotalCap fn() with llap_daemon_selected_queue_name= 'default'.2018-02-06 11:44:14,737 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity2018-02-06 11:44:14,737 - Queue list : ['root', 'default']2018-02-06 11:44:14,737 - DBG: Selected queue name as: yarn.scheduler.capacity.root.capacity2018-02-06 11:44:14,737 - Total capacity available for queue root is : 49152.02018-02-06 11:44:14,738 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity2018-02-06 11:44:14,738 - Total capacity available for queue default is : 4915.22018-02-06 11:44:14,738 - DBG: Calculated 'default' queue available capacity : 4096.0, using following: llap_daemon_selected_queue_cap : 4915.2, yarn_min_container_size : 2048.02018-02-06 11:44:14,738 - DBG: Calculated 'num_llap_nodes_requested' : 1.0, using following: total_llap_mem_normalized : 4096.0, yarn_nm_mem_in_mb_normalized : 24576.02018-02-06 11:44:14,738 - Setting config 'num_llap_nodes' as : 1.02018-02-06 11:44:14,738 - Returning default AM percent value : '0.1' for queue : default2018-02-06 11:44:14,738 - DBG: Calculated 'hive_tez_am_cap_available' : 409.6, using following: queue_am_fraction_perc : 0.1, total_llap_mem_normalized : 4096.02018-02-06 11:44:14,738 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 2048.0, using following : total_llap_mem_normalized : 4096.0, slider_am_container_size : 2048.02018-02-06 11:44:14,738 - Not enough capacity available on the cluster to run LLAP2018-02-06 11:44:14,738 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration2018-02-06 11:44:14,739 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 20482018-02-06 11:44:14,739 - Setting Yarn Repo user for Ranger.2018-02-06 11:44:14,739 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,739 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,740 - No oozie configurations available2018-02-06 11:44:14,742 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,742 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,742 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,742 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,743 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,744 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,745 - Updated 'tez-site' config 'tez.task.launch.cmd-opts' and 'tez.am.launch.cmd-opts' as : -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC{{heap_dump_opts}}2018-02-06 11:44:14,745 - call type context : {'call_type': 'recommendConfigurationDependencies'}2018-02-06 11:44:14,745 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,746 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,746 - DBG: Entered updateLlapConfigs2018-02-06 11:44:14,746 - Is cluster create operation ? = False2018-02-06 11:44:14,746 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.2018-02-06 11:44:14,746 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.2018-02-06 11:44:14,747 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,747 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,747 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.2018-02-06 11:44:14,747 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'hunger_games', 'label': 'hunger_games'}, {'value': 'load_raw', 'label': 'load_raw'}, {'value': 'low_prior', 'label': 'low_prior'}, {'value': 'queries', 'label': 'queries'}]2018-02-06 11:44:14,747 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 772018-02-06 11:44:14,747 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]2018-02-06 11:44:14,748 - YARN leaf Queues = set(['default', 'hunger_games', 'low_prior', 'load_raw', 'queries'])2018-02-06 11:44:14,748 - DBG: llap_named_queue_selected_in_curr_invocation = False2018-02-06 11:44:14,748 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'True'.2018-02-06 11:44:14,748 - Selected YARN queue for LLAP is : 'default'. Current YARN queues : ['default', 'hunger_games', 'low_prior', 'load_raw', 'queries']. Setting 'Number of LLAP nodes' visibility to 'False'.2018-02-06 11:44:14,748 - DBG: Performing LLAP config calculations ......2018-02-06 11:44:14,748 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.02018-02-06 11:44:14,748 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 20482018-02-06 11:44:14,748 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 10242018-02-06 11:44:14,748 - DBG Calculated normalized_tez_am_container_size : 2048.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.02018-02-06 11:44:14,749 - DBG: Calculated and returning 'hive_tez_container_size' : 30722018-02-06 11:44:14,749 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.02018-02-06 11:44:14,749 - DBG: Calculated 'slider_am_container_size' : 2048.0, using following: yarn_min_container_size : 2048.02018-02-06 11:44:14,749 - DBG: Calculated 'min_memory_required': 8192.0 using following : slider_am_container_size: 2048.0, normalized_tez_am_container_size : 2048.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 2048.02018-02-06 11:44:14,749 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 8192.0, yarn_nm_mem_in_mb_normalized : 24576.02018-02-06 11:44:14,749 - DBG: selected_queue_is_ambari_managed_llap = False2018-02-06 11:44:14,749 - Entered __getSelectedQueueTotalCap fn() with llap_daemon_selected_queue_name= 'default'.2018-02-06 11:44:14,749 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity2018-02-06 11:44:14,749 - Queue list : ['root', 'default']2018-02-06 11:44:14,749 - DBG: Selected queue name as: yarn.scheduler.capacity.root.capacity2018-02-06 11:44:14,749 - Total capacity available for queue root is : 49152.02018-02-06 11:44:14,750 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity2018-02-06 11:44:14,750 - Total capacity available for queue default is : 4915.22018-02-06 11:44:14,750 - DBG: Calculated 'default' queue available capacity : 4096.0, using following: llap_daemon_selected_queue_cap : 4915.2, yarn_min_container_size : 2048.02018-02-06 11:44:14,750 - DBG: Calculated 'num_llap_nodes_requested' : 1.0, using following: total_llap_mem_normalized : 4096.0, yarn_nm_mem_in_mb_normalized : 24576.02018-02-06 11:44:14,750 - Setting config 'num_llap_nodes' as : 1.02018-02-06 11:44:14,750 - Returning default AM percent value : '0.1' for queue : default2018-02-06 11:44:14,750 - DBG: Calculated 'hive_tez_am_cap_available' : 409.6, using following: queue_am_fraction_perc : 0.1, total_llap_mem_normalized : 4096.02018-02-06 11:44:14,750 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 2048.0, using following : total_llap_mem_normalized : 4096.0, slider_am_container_size : 2048.02018-02-06 11:44:14,750 - Not enough capacity available on the cluster to run LLAP2018-02-06 11:44:14,750 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration2018-02-06 11:44:14,751 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 20482018-02-06 11:44:14,751 - Setting Yarn Repo user for Ranger.06 Feb 2018 11:44:14,820  INFO [ambari-client-thread-75953] StackAdvisorRunner:71 -  advisor script stderr: 
4 REPLIES 4

Re: Ambari doesn't install llap

@Ruslan Fialkovsky check your min-mb allocation in the YARN. It looks like LLAP can't allocate the memory it needs to start.

Not enough capacity available on the cluster to run LLAP2018-02-06 11:44:14,750 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration2018-02-06 11:44:14,751 - DBG: 'yarn.scheduler.minimum-allocation-mb

Re: Ambari doesn't install llap

Expert Contributor

@Scott Shaw

I have set the default settings for YARN queue.
Resources is enough, but Installation Wizard is still does not show list hosts for LLAP.

56604-4p9md.png

07 Feb 2018 13:36:49,138  INFO [ambari-client-thread-85671] StackAdvisorRunner:71 -     advisor script stderr: 
07 Feb 2018 13:37:18,634  INFO [ambari-client-thread-85949] StackAdvisorRunner:47 - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, actionDirectory=/var/run/ambari-server/stack-recommendations/45, command=recommend-configuration-dependencies
07 Feb 2018 13:37:18,642  INFO [ambari-client-thread-85949] StackAdvisorRunner:61 - Stack-advisor output=/var/run/ambari-server/stack-recommendations/45/stackadvisor.out, error=/var/run/ambari-server/stack-recommendations/45/stackadvisor.err
07 Feb 2018 13:37:19,555  INFO [ambari-client-thread-85949] StackAdvisorRunner:69 - Stack advisor output files
07 Feb 2018 13:37:19,556  INFO [ambari-client-thread-85949] StackAdvisorRunner:70 -     advisor script stdout: StackAdvisor implementation for stack HDP, version 2.0.6 was loaded
StackAdvisor implementation for stack HDP, version 2.1 was loaded
StackAdvisor implementation for stack HDP, version 2.2 was loaded
StackAdvisor implementation for stack HDP, version 2.3 was loaded
StackAdvisor implementation for stack HDP, version 2.4 was loaded
StackAdvisor implementation for stack HDP, version 2.5 was loaded
StackAdvisor implementation for stack HDP, version 2.6 was loaded
Returning HDP26StackAdvisor implementation
2018-02-07 13:37:19,440 - Memory for YARN apps - cluster[totalAvailableRam]: 17408
2018-02-07 13:37:19,441 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-07 13:37:19,441 - user operation context : EditConfig
2018-02-07 13:37:19,441 - Full context: callContext = recommendConfigurationDependencies and operation = EditConfig and adding YARN = False and old value exists = None
2018-02-07 13:37:19,441 - Using user provided yarn.scheduler.minimum-allocation-mb = 1024
2018-02-07 13:37:19,441 - Minimum ram per container due to user input - cluster[yarnMinContainerSize]: 1024
2018-02-07 13:37:19,441 - Containers per node - cluster[containers]: 8
2018-02-07 13:37:19,441 - Ram per containers before normalization - cluster[ramPerContainer]: 2176
2018-02-07 13:37:19,441 - Min container size - cluster[yarnMinContainerSize]: 1024
2018-02-07 13:37:19,442 - Available memory for map - cluster[mapMemory]: 2048
2018-02-07 13:37:19,442 - Available memory for reduce - cluster[reduceMemory]: 2048
2018-02-07 13:37:19,442 - Available memory for am - cluster[amMemory]: 2048
ServiceAdvisor implementation for service NIFI was loaded
Failed to load or create ServiceAdvisor implementation for service REGISTRY: Expecting class name REGISTRY030ServiceAdvisor but it was not found.
Failed to load or create ServiceAdvisor implementation for service STREAMLINE: Expecting class name STREAMLINE050ServiceAdvisor but it was not found.
max_inmemory_regions: -2.9
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/AMBARI_METRICS.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HBASE.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HDFS.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HIVE.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HOST.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/STORM.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/YARN.txt
metrics length: 333
2018-02-07 13:37:19,450 - Constructing atlas.rest.address=http://.*.*.*:21000
2018-02-07 13:37:19,453 - Setting Hbase Repo user for Ranger.
2018-02-07 13:37:19,454 - Hive Server and Hive Server Interactive (if enabled) Host List : set([u'.*.*.*'])
2018-02-07 13:37:19,454 - Updated hadoop.proxyuser.hive.hosts as : .*.*.*
2018-02-07 13:37:19,454 - Updated hadoop.proxyuser.hive.hosts as : *
2018-02-07 13:37:19,455 - Updated hadoop.proxyuser.hive.hosts as : *
2018-02-07 13:37:19,455 - Updated hadoop.proxyuser.hive.hosts as : .*.*.*
2018-02-07 13:37:19,455 - Updated hadoop.proxyuser.hive.hosts as : *
2018-02-07 13:37:19,456 - Setting HDFS Repo user for Ranger.
2018-02-07 13:37:19,456 - DBG: Invoked recommendHiveConfiguration
2018-02-07 13:37:19,457 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-07 13:37:19,457 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,458 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,458 - DBG: Entered updateLlapConfigs
2018-02-07 13:37:19,458 - Is cluster create operation ? = False
2018-02-07 13:37:19,458 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-07 13:37:19,458 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.
2018-02-07 13:37:19,459 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,459 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,459 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.
2018-02-07 13:37:19,459 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'llap', 'label': 'llap'}]
2018-02-07 13:37:19,459 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,459 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,459 - YARN leaf Queues = set(['default', 'llap'])
2018-02-07 13:37:19,459 - DBG: llap_named_queue_selected_in_curr_invocation = True
2018-02-07 13:37:19,459 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'False'.
2018-02-07 13:37:19,460 - DBG: Selected YARN queue for LLAP is : 'llap'. Current YARN queues : ['default', 'llap']. Setting 'Number of LLAP nodes' slider visibility to 'True'
2018-02-07 13:37:19,460 - 
DBG: Performing LLAP config calculations ......
2018-02-07 13:37:19,460 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.0
2018-02-07 13:37:19,460 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 1024
2018-02-07 13:37:19,460 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 1024
2018-02-07 13:37:19,460 - DBG Calculated normalized_tez_am_container_size : 1024.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.0
2018-02-07 13:37:19,460 - DBG: Calculated and returning 'hive_tez_container_size' : 3072
2018-02-07 13:37:19,460 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0
2018-02-07 13:37:19,460 - DBG: Calculated 'slider_am_container_size' : 1024.0, using following: yarn_min_container_size : 1024.0
2018-02-07 13:37:19,461 - DBG: Calculated 'min_memory_required': 5120.0 using following : slider_am_container_size: 1024.0, normalized_tez_am_container_size : 1024.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 1024.0
2018-02-07 13:37:19,461 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 5120.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-07 13:37:19,461 - DBG: selected_queue_is_ambari_managed_llap = True
2018-02-07 13:37:19,461 - DBG: Calculated 'total_llap_mem' : 24576.0, using following: num_llap_nodes_requested : 1, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-07 13:37:19,461 - DBG: Calculated 'total_llap_mem_normalized' : 24576.0, using following: total_llap_mem : 24576.0, yarn_min_container_size : 1024.0
2018-02-07 13:37:19,461 - DBG: Calculated 'llap' queue capacity percent = 50.0.
2018-02-07 13:37:19,461 - Determining creation/adjustment of 'capacity-scheduler' for 'llap' queue.
2018-02-07 13:37:19,461 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,461 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,462 - Updated 'capacity-scheduler' configs as one concatenated string.
2018-02-07 13:37:19,462 - Adjusted YARN Queue : 'llap'. Current capacity : 50.0%. State: RUNNING.
2018-02-07 13:37:19,462 - Adjusted 'default' queue capacity to : 50.0%
2018-02-07 13:37:19,462 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-07 13:37:19,462 - 'capacity-scheduler' configs is set as a single '\n' separated string in current invocation. count(configurations['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,462 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'llap', 'label': 'llap'}]
2018-02-07 13:37:19,462 - DBG: hive_tez_am_cap_available : 24576.0
2018-02-07 13:37:19,462 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 23552.0, using following : total_llap_mem_normalized : 24576.0, slider_am_container_size : 1024.0
2018-02-07 13:37:19,462 - DBG: Calculated 'max_executors_per_node' : 8.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,462 - DBG: Calculated 'llap_concurreny_limit' : 0.0, using following : max_executors_per_node : 8.0, num_llap_nodes_requested : 1, DEFAULT_EXECUTOR_TO_AM_RATIO : 20, MAX_CONCURRENT_QUERIES : 32
2018-02-07 13:37:19,463 - DBG: Calculated 'llap_concurrency' : 0.0, using following : llap_concurreny_limit : 0.0, llap_mem_for_tezAm_and_daemons : 23552.0, DEFAULT_EXECUTOR_TO_AM_RATIO : 20, mem_per_thread_for_llap : 3072.0, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,463 - DBG: Readjusted 'llap_concurrency' to : 1. Earlier calculated value : 0
2018-02-07 13:37:19,463 - DBG: Calculated 'max_llap_concurreny_limit' : 0.0, using following : max_executors_per_node : 8.0, num_llap_nodes_requested : 1, MIN_EXECUTOR_TO_AM_RATIO : 10, MAX_CONCURRENT_QUERIES : 32
2018-02-07 13:37:19,463 - DBG: Calculated 'max_llap_concurreny' : 0, using following : max_llap_concurreny_limit : 0.0, llap_mem_for_tezAm_and_daemons : 23552.0, MIN_EXECUTOR_TO_AM_RATIO : 10, mem_per_thread_for_llap : 3072.0, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,463 - DBG: Adjusting 'max_llap_concurreny' from 0 to 4
2018-02-07 13:37:19,463 - DBG: Calculated 'tez_am_memory_required' : 1024.0, using following : llap_concurrency : 1, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,463 - DBG: Calculated 'llap_mem_daemon_size' : 22528.0, using following : llap_mem_for_tezAm_and_daemons : 23552.0, tez_am_memory_required : 1024.0
2018-02-07 13:37:19,463 - DBG: Calculated 'llap_daemon_mem_per_node' : 22528.0, using following : llap_mem_daemon_size : 22528.0, num_llap_nodes_requested : 1, yarn_min_container_size: 1024.0
2018-02-07 13:37:19,463 - DBG: num_llap_nodes : 1
2018-02-07 13:37:19,463 - DBG: Determined final memPerDaemon: 22528.0, using following: concurrency: 1, numNMNodes: 2, numLlapNodes: 1 
2018-02-07 13:37:19,464 - DBG: Calculated 'num_executors_per_node_max' : 8.0, using following : yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,464 - DBG: Calculated 'num_executors_per_node' : 7.0, using following : llap_daemon_mem_per_node : 22528.0, num_executors_per_node_max : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,464 - DBG: Calculated 'Cache per node' : 1024.0, using following : llap_daemon_mem_per_node : 22528.0, total_mem_for_executors_per_node : 21504.0
2018-02-07 13:37:19,464 - 'llap_headroom_space' read from services as : 6144.0
2018-02-07 13:37:19,464 - DBG: Calculated llap_app_heap_size : 17203.2, using following : total_mem_for_executors : 21504.0
2018-02-07 13:37:19,464 - DBG: Applying the calculated values....
2018-02-07 13:37:19,464 - DBG: Setting 'tez.am.resource.memory.mb' config value as : 1024
2018-02-07 13:37:19,464 - DBG: Used num_llap_nodes for calculations : 1
2018-02-07 13:37:19,464 - DBG: Setting config 'num_llap_nodes_for_llap_daemons' as : 1
2018-02-07 13:37:19,464 - DBG: Setting 'hive.tez.container.size' config value as : 3072
2018-02-07 13:37:19,465 - DBG: Putting num_executors_per_node as 7
2018-02-07 13:37:19,465 - DBG: Done putting all configs
2018-02-07 13:37:19,465 - Setting Yarn Repo user for Ranger.
2018-02-07 13:37:19,465 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,465 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,466 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'True'.
2018-02-07 13:37:19,466 - Setting Hive Repo user for Ranger.
2018-02-07 13:37:19,467 - Not setting Kafka Repo user for Ranger.
2018-02-07 13:37:19,467 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-07 13:37:19,468 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,468 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,468 - DBG: Entered updateLlapConfigs
2018-02-07 13:37:19,468 - Is cluster create operation ? = False
2018-02-07 13:37:19,468 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-07 13:37:19,469 - 'capacity-scheduler' configs is set as a single '\n' separated string in current invocation. count(configurations['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,469 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'llap', 'label': 'llap'}]
2018-02-07 13:37:19,469 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,469 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,469 - YARN leaf Queues = set(['default', 'llap'])
2018-02-07 13:37:19,469 - DBG: llap_named_queue_selected_in_curr_invocation = True
2018-02-07 13:37:19,469 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'False'.
2018-02-07 13:37:19,469 - DBG: Selected YARN queue for LLAP is : 'llap'. Current YARN queues : ['default', 'llap']. Setting 'Number of LLAP nodes' slider visibility to 'True'
2018-02-07 13:37:19,469 - 
DBG: Performing LLAP config calculations ......
2018-02-07 13:37:19,470 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.0
2018-02-07 13:37:19,470 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 1024
2018-02-07 13:37:19,470 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 1024
2018-02-07 13:37:19,470 - DBG Calculated normalized_tez_am_container_size : 1024.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.0
2018-02-07 13:37:19,470 - DBG: Calculated and returning 'hive_tez_container_size' : 3072
2018-02-07 13:37:19,470 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0
2018-02-07 13:37:19,470 - DBG: Calculated 'slider_am_container_size' : 1024.0, using following: yarn_min_container_size : 1024.0
2018-02-07 13:37:19,470 - DBG: Calculated 'min_memory_required': 5120.0 using following : slider_am_container_size: 1024.0, normalized_tez_am_container_size : 1024.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 1024.0
2018-02-07 13:37:19,470 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 5120.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-07 13:37:19,470 - DBG: selected_queue_is_ambari_managed_llap = True
2018-02-07 13:37:19,471 - DBG: Calculated 'total_llap_mem' : 24576.0, using following: num_llap_nodes_requested : 1, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-07 13:37:19,471 - DBG: Calculated 'total_llap_mem_normalized' : 24576.0, using following: total_llap_mem : 24576.0, yarn_min_container_size : 1024.0
2018-02-07 13:37:19,471 - DBG: Calculated 'llap' queue capacity percent = 50.0.
2018-02-07 13:37:19,471 - Determining creation/adjustment of 'capacity-scheduler' for 'llap' queue.
2018-02-07 13:37:19,471 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,471 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,471 - Updated 'capacity-scheduler' configs as one concatenated string.
2018-02-07 13:37:19,471 - Adjusted YARN Queue : 'llap'. Current capacity : 50.0%. State: RUNNING.
2018-02-07 13:37:19,472 - Adjusted 'default' queue capacity to : 50.0%
2018-02-07 13:37:19,472 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-07 13:37:19,472 - 'capacity-scheduler' configs is set as a single '\n' separated string in current invocation. count(configurations['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,472 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'llap', 'label': 'llap'}]
2018-02-07 13:37:19,472 - DBG: hive_tez_am_cap_available : 24576.0
2018-02-07 13:37:19,472 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 23552.0, using following : total_llap_mem_normalized : 24576.0, slider_am_container_size : 1024.0
2018-02-07 13:37:19,472 - DBG: Calculated 'max_executors_per_node' : 8.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,472 - DBG: Calculated 'llap_concurreny_limit' : 0.0, using following : max_executors_per_node : 8.0, num_llap_nodes_requested : 1, DEFAULT_EXECUTOR_TO_AM_RATIO : 20, MAX_CONCURRENT_QUERIES : 32
2018-02-07 13:37:19,472 - DBG: Calculated 'llap_concurrency' : 0.0, using following : llap_concurreny_limit : 0.0, llap_mem_for_tezAm_and_daemons : 23552.0, DEFAULT_EXECUTOR_TO_AM_RATIO : 20, mem_per_thread_for_llap : 3072.0, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,472 - DBG: Readjusted 'llap_concurrency' to : 1. Earlier calculated value : 0
2018-02-07 13:37:19,473 - DBG: Calculated 'max_llap_concurreny_limit' : 0.0, using following : max_executors_per_node : 8.0, num_llap_nodes_requested : 1, MIN_EXECUTOR_TO_AM_RATIO : 10, MAX_CONCURRENT_QUERIES : 32
2018-02-07 13:37:19,473 - DBG: Calculated 'max_llap_concurreny' : 0, using following : max_llap_concurreny_limit : 0.0, llap_mem_for_tezAm_and_daemons : 23552.0, MIN_EXECUTOR_TO_AM_RATIO : 10, mem_per_thread_for_llap : 3072.0, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,473 - DBG: Adjusting 'max_llap_concurreny' from 0 to 4
2018-02-07 13:37:19,473 - DBG: Calculated 'tez_am_memory_required' : 1024.0, using following : llap_concurrency : 1, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,473 - DBG: Calculated 'llap_mem_daemon_size' : 22528.0, using following : llap_mem_for_tezAm_and_daemons : 23552.0, tez_am_memory_required : 1024.0
2018-02-07 13:37:19,473 - DBG: Calculated 'llap_daemon_mem_per_node' : 22528.0, using following : llap_mem_daemon_size : 22528.0, num_llap_nodes_requested : 1, yarn_min_container_size: 1024.0
2018-02-07 13:37:19,473 - DBG: num_llap_nodes : 1
2018-02-07 13:37:19,473 - DBG: Determined final memPerDaemon: 22528.0, using following: concurrency: 1, numNMNodes: 2, numLlapNodes: 1 
2018-02-07 13:37:19,473 - DBG: Calculated 'num_executors_per_node_max' : 8.0, using following : yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,473 - DBG: Calculated 'num_executors_per_node' : 7.0, using following : llap_daemon_mem_per_node : 22528.0, num_executors_per_node_max : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,473 - DBG: Calculated 'Cache per node' : 1024.0, using following : llap_daemon_mem_per_node : 22528.0, total_mem_for_executors_per_node : 21504.0
2018-02-07 13:37:19,474 - 'llap_headroom_space' read from services as : 6144.0
2018-02-07 13:37:19,474 - DBG: Calculated llap_app_heap_size : 17203.2, using following : total_mem_for_executors : 21504.0
2018-02-07 13:37:19,474 - DBG: Applying the calculated values....
2018-02-07 13:37:19,474 - DBG: Setting 'tez.am.resource.memory.mb' config value as : 1024
2018-02-07 13:37:19,474 - DBG: Used num_llap_nodes for calculations : 1
2018-02-07 13:37:19,474 - DBG: Setting config 'num_llap_nodes_for_llap_daemons' as : 1
2018-02-07 13:37:19,474 - DBG: Setting 'hive.tez.container.size' config value as : 3072
2018-02-07 13:37:19,474 - DBG: Putting num_executors_per_node as 7
2018-02-07 13:37:19,474 - DBG: Done putting all configs
2018-02-07 13:37:19,475 - Setting Yarn Repo user for Ranger.
2018-02-07 13:37:19,475 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,475 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,476 - No oozie configurations available
2018-02-07 13:37:19,477 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,477 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,478 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,478 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,479 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,479 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,480 - Updated 'tez-site' config 'tez.task.launch.cmd-opts' and 'tez.am.launch.cmd-opts' as : -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC{{heap_dump_opts}}
2018-02-07 13:37:19,480 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-07 13:37:19,480 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,481 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,481 - DBG: Entered updateLlapConfigs
2018-02-07 13:37:19,481 - Is cluster create operation ? = False
2018-02-07 13:37:19,481 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-07 13:37:19,481 - 'capacity-scheduler' configs is set as a single '\n' separated string in current invocation. count(configurations['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,482 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'llap', 'label': 'llap'}]
2018-02-07 13:37:19,482 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,482 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,482 - YARN leaf Queues = set(['default', 'llap'])
2018-02-07 13:37:19,482 - DBG: llap_named_queue_selected_in_curr_invocation = True
2018-02-07 13:37:19,482 - DBG: Setting 'num_llap_nodes' config's  READ ONLY attribute as 'False'.
2018-02-07 13:37:19,482 - DBG: Selected YARN queue for LLAP is : 'llap'. Current YARN queues : ['default', 'llap']. Setting 'Number of LLAP nodes' slider visibility to 'True'
2018-02-07 13:37:19,482 - 
DBG: Performing LLAP config calculations ......
2018-02-07 13:37:19,483 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.0
2018-02-07 13:37:19,483 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 1024
2018-02-07 13:37:19,483 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 1024
2018-02-07 13:37:19,483 - DBG Calculated normalized_tez_am_container_size : 1024.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.0
2018-02-07 13:37:19,483 - DBG: Calculated and returning 'hive_tez_container_size' : 3072
2018-02-07 13:37:19,483 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0
2018-02-07 13:37:19,483 - DBG: Calculated 'slider_am_container_size' : 1024.0, using following: yarn_min_container_size : 1024.0
2018-02-07 13:37:19,483 - DBG: Calculated 'min_memory_required': 5120.0 using following : slider_am_container_size: 1024.0, normalized_tez_am_container_size : 1024.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 1024.0
2018-02-07 13:37:19,483 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 5120.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-07 13:37:19,483 - DBG: selected_queue_is_ambari_managed_llap = True
2018-02-07 13:37:19,483 - DBG: Calculated 'total_llap_mem' : 24576.0, using following: num_llap_nodes_requested : 1, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-07 13:37:19,484 - DBG: Calculated 'total_llap_mem_normalized' : 24576.0, using following: total_llap_mem : 24576.0, yarn_min_container_size : 1024.0
2018-02-07 13:37:19,484 - DBG: Calculated 'llap' queue capacity percent = 50.0.
2018-02-07 13:37:19,484 - Determining creation/adjustment of 'capacity-scheduler' for 'llap' queue.
2018-02-07 13:37:19,484 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,484 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.llap.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.llap.maximum-capacity', '0'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.llap.acl_submit_applications', 'hive'), ('yarn.scheduler.capacity.root.llap.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.priority', '1'), ('yarn.scheduler.capacity.root.queues', 'default,llap'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.llap.capacity', '0'), ('yarn.scheduler.capacity.root.llap.acl_administer_queue', 'hive'), ('yarn.scheduler.capacity.root.llap.maximum-am-resource-percent', '1'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.llap.priority', '10'), ('yarn.scheduler.capacity.root.llap.user-limit-factor', '1'), ('yarn.scheduler.capacity.root.llap.state', 'STOPPED'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.default.capacity', '100'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.resource-calculator', 'org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator')]
2018-02-07 13:37:19,484 - Updated 'capacity-scheduler' configs as one concatenated string.
2018-02-07 13:37:19,485 - Adjusted YARN Queue : 'llap'. Current capacity : 50.0%. State: RUNNING.
2018-02-07 13:37:19,485 - Adjusted 'default' queue capacity to : 50.0%
2018-02-07 13:37:19,485 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-07 13:37:19,485 - 'capacity-scheduler' configs is set as a single '\n' separated string in current invocation. count(configurations['capacity-scheduler']['properties']['capacity-scheduler']) = 27
2018-02-07 13:37:19,485 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'llap', 'label': 'llap'}]
2018-02-07 13:37:19,485 - DBG: hive_tez_am_cap_available : 24576.0
2018-02-07 13:37:19,485 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 23552.0, using following : total_llap_mem_normalized : 24576.0, slider_am_container_size : 1024.0
2018-02-07 13:37:19,485 - DBG: Calculated 'max_executors_per_node' : 8.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,485 - DBG: Calculated 'llap_concurreny_limit' : 0.0, using following : max_executors_per_node : 8.0, num_llap_nodes_requested : 1, DEFAULT_EXECUTOR_TO_AM_RATIO : 20, MAX_CONCURRENT_QUERIES : 32
2018-02-07 13:37:19,486 - DBG: Calculated 'llap_concurrency' : 0.0, using following : llap_concurreny_limit : 0.0, llap_mem_for_tezAm_and_daemons : 23552.0, DEFAULT_EXECUTOR_TO_AM_RATIO : 20, mem_per_thread_for_llap : 3072.0, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,486 - DBG: Readjusted 'llap_concurrency' to : 1. Earlier calculated value : 0
2018-02-07 13:37:19,486 - DBG: Calculated 'max_llap_concurreny_limit' : 0.0, using following : max_executors_per_node : 8.0, num_llap_nodes_requested : 1, MIN_EXECUTOR_TO_AM_RATIO : 10, MAX_CONCURRENT_QUERIES : 32
2018-02-07 13:37:19,486 - DBG: Calculated 'max_llap_concurreny' : 0, using following : max_llap_concurreny_limit : 0.0, llap_mem_for_tezAm_and_daemons : 23552.0, MIN_EXECUTOR_TO_AM_RATIO : 10, mem_per_thread_for_llap : 3072.0, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,486 - DBG: Adjusting 'max_llap_concurreny' from 0 to 4
2018-02-07 13:37:19,486 - DBG: Calculated 'tez_am_memory_required' : 1024.0, using following : llap_concurrency : 1, normalized_tez_am_container_size : 1024.0
2018-02-07 13:37:19,486 - DBG: Calculated 'llap_mem_daemon_size' : 22528.0, using following : llap_mem_for_tezAm_and_daemons : 23552.0, tez_am_memory_required : 1024.0
2018-02-07 13:37:19,486 - DBG: Calculated 'llap_daemon_mem_per_node' : 22528.0, using following : llap_mem_daemon_size : 22528.0, num_llap_nodes_requested : 1, yarn_min_container_size: 1024.0
2018-02-07 13:37:19,486 - DBG: num_llap_nodes : 1
2018-02-07 13:37:19,486 - DBG: Determined final memPerDaemon: 22528.0, using following: concurrency: 1, numNMNodes: 2, numLlapNodes: 1 
2018-02-07 13:37:19,486 - DBG: Calculated 'num_executors_per_node_max' : 8.0, using following : yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,487 - DBG: Calculated 'num_executors_per_node' : 7.0, using following : llap_daemon_mem_per_node : 22528.0, num_executors_per_node_max : 8.0, mem_per_thread_for_llap: 3072.0
2018-02-07 13:37:19,487 - DBG: Calculated 'Cache per node' : 1024.0, using following : llap_daemon_mem_per_node : 22528.0, total_mem_for_executors_per_node : 21504.0
2018-02-07 13:37:19,487 - 'llap_headroom_space' read from services as : 6144.0
2018-02-07 13:37:19,487 - DBG: Calculated llap_app_heap_size : 17203.2, using following : total_mem_for_executors : 21504.0
2018-02-07 13:37:19,487 - DBG: Applying the calculated values....
2018-02-07 13:37:19,487 - DBG: Setting 'tez.am.resource.memory.mb' config value as : 1024
2018-02-07 13:37:19,487 - DBG: Used num_llap_nodes for calculations : 1
2018-02-07 13:37:19,487 - DBG: Setting config 'num_llap_nodes_for_llap_daemons' as : 1
2018-02-07 13:37:19,487 - DBG: Setting 'hive.tez.container.size' config value as : 3072
2018-02-07 13:37:19,488 - DBG: Putting num_executors_per_node as 7
2018-02-07 13:37:19,488 - DBG: Done putting all configs
2018-02-07 13:37:19,488 - Setting Yarn Repo user for Ranger.
07 Feb 2018 13:37:19,556  INFO [ambari-client-thread-85949] StackAdvisorRunner:71 -     advisor script stderr: 
07 Feb 2018 13:39:12,859  INFO [ambari-client-thread-85908] QuickLinkVisibilityControllerFactory:44 - No quick link profile is set, will display all quicklinks.
07 Feb 2018 13:39:13,797  INFO [ambari-client-thread-85948] QuickLinkVisibilityControllerFactory:44 - No quick link profile is set, will display all quicklinks.
07 Feb 2018 13:39:13,805  INFO [ambari-client-thread-85908] QuickLinkVisibilityControllerFactory:44 - No quick link profile is set, will display all quicklinks.

Re: Ambari doesn't install llap

Expert Contributor

What does it mean :

07 Feb 2018 13:37:19,556  INFO [ambari-client-thread-85949] StackAdvisorRunner:71 -     advisor script stderr: 
07 Feb 2018 13:39:12,859  INFO [ambari-client-thread-85908] QuickLinkVisibilityControllerFactory:44 - No quick link profile is set, will display all quicklinks.

Why "advisor script stderr:" is empty?

Highlighted

Re: Ambari doesn't install llap

Expert Contributor

Problem was resolved after ambari was upgraded.

Don't have an account?
Coming from Hortonworks? Activate your account here