06 Feb 2018 11:44:13,929 INFO [ambari-client-thread-75953] StackAdvisorRunner:47 - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, actionDirectory=/var/run/ambari-server/stack-recommendations/37, command=recommend-configuration-dependencies
06 Feb 2018 11:44:13,937 INFO [ambari-client-thread-75953] StackAdvisorRunner:61 - Stack-advisor output=/var/run/ambari-server/stack-recommendations/37/stackadvisor.out, error=/var/run/ambari-server/stack-recommendations/37/stackadvisor.err
06 Feb 2018 11:44:14,819 INFO [ambari-client-thread-75953] StackAdvisorRunner:69 - Stack advisor output files
06 Feb 2018 11:44:14,820 INFO [ambari-client-thread-75953] StackAdvisorRunner:70 - advisor script stdout: StackAdvisor implementation for stack HDP, version 2.0.6 was loaded
StackAdvisor implementation for stack HDP, version 2.1 was loaded
StackAdvisor implementation for stack HDP, version 2.2 was loaded
StackAdvisor implementation for stack HDP, version 2.3 was loaded
StackAdvisor implementation for stack HDP, version 2.4 was loaded
StackAdvisor implementation for stack HDP, version 2.5 was loaded
StackAdvisor implementation for stack HDP, version 2.6 was loaded
Returning HDP26StackAdvisor implementation
2018-02-06 11:44:14,707 - Memory for YARN apps - cluster[totalAvailableRam]: 17408
2018-02-06 11:44:14,707 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-06 11:44:14,707 - user operation context : EditConfig
2018-02-06 11:44:14,707 - Full context: callContext = recommendConfigurationDependencies and operation = EditConfig and adding YARN = False and old value exists = None
2018-02-06 11:44:14,708 - Using user provided yarn.scheduler.minimum-allocation-mb = 2048
2018-02-06 11:44:14,708 - Minimum ram per container due to user input - cluster[yarnMinContainerSize]: 2048
2018-02-06 11:44:14,708 - Containers per node - cluster[containers]: 8
2018-02-06 11:44:14,708 - Ram per containers before normalization - cluster[ramPerContainer]: 2176
2018-02-06 11:44:14,708 - Min container size - cluster[yarnMinContainerSize]: 2048
2018-02-06 11:44:14,708 - Available memory for map - cluster[mapMemory]: 2048
2018-02-06 11:44:14,708 - Available memory for reduce - cluster[reduceMemory]: 2048
2018-02-06 11:44:14,708 - Available memory for am - cluster[amMemory]: 2048
ServiceAdvisor implementation for service NIFI was loaded
Failed to load or create ServiceAdvisor implementation for service REGISTRY: Expecting class name REGISTRY030ServiceAdvisor but it was not found.
Failed to load or create ServiceAdvisor implementation for service STREAMLINE: Expecting class name STREAMLINE050ServiceAdvisor but it was not found.
max_inmemory_regions: -2.9
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/AMBARI_METRICS.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HBASE.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HDFS.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HIVE.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/HOST.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/STORM.txt
Processing file: /var/lib/ambari-server/resources/stacks/HDP/2.6/services/../../../../common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/YARN.txt
metrics length: 333
2018-02-06 11:44:14,717 - Constructing atlas.rest.address=http://*.*.*:21000
2018-02-06 11:44:14,720 - Setting Hbase Repo user for Ranger.
2018-02-06 11:44:14,721 - Hive Server and Hive Server Interactive (if enabled) Host List : set([u'*.*.*'])
2018-02-06 11:44:14,721 - Updated hadoop.proxyuser.hive.hosts as : *.*.*
2018-02-06 11:44:14,721 - Updated hadoop.proxyuser.hive.hosts as : *
2018-02-06 11:44:14,722 - Updated hadoop.proxyuser.hive.hosts as : *
2018-02-06 11:44:14,722 - Updated hadoop.proxyuser.hive.hosts as : *.*.*
2018-02-06 11:44:14,722 - Updated hadoop.proxyuser.hive.hosts as : *
2018-02-06 11:44:14,723 - Setting HDFS Repo user for Ranger.
2018-02-06 11:44:14,723 - DBG: Invoked recommendHiveConfiguration
2018-02-06 11:44:14,724 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-06 11:44:14,724 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,725 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,725 - DBG: Entered updateLlapConfigs
2018-02-06 11:44:14,725 - Is cluster create operation ? = False
2018-02-06 11:44:14,725 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-06 11:44:14,726 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.
2018-02-06 11:44:14,726 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,726 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,726 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.
2018-02-06 11:44:14,726 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'hunger_games', 'label': 'hunger_games'}, {'value': 'load_raw', 'label': 'load_raw'}, {'value': 'low_prior', 'label': 'low_prior'}, {'value': 'queries', 'label': 'queries'}]
2018-02-06 11:44:14,726 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,727 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,727 - YARN leaf Queues = set(['default', 'hunger_games', 'low_prior', 'load_raw', 'queries'])
2018-02-06 11:44:14,727 - DBG: llap_named_queue_selected_in_curr_invocation = False
2018-02-06 11:44:14,727 - DBG: Setting 'num_llap_nodes' config's READ ONLY attribute as 'True'.
2018-02-06 11:44:14,727 - Selected YARN queue for LLAP is : 'default'. Current YARN queues : ['default', 'hunger_games', 'low_prior', 'load_raw', 'queries']. Setting 'Number of LLAP nodes' visibility to 'False'.
2018-02-06 11:44:14,727 -
DBG: Performing LLAP config calculations ......
2018-02-06 11:44:14,727 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.0
2018-02-06 11:44:14,727 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 2048
2018-02-06 11:44:14,727 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 1024
2018-02-06 11:44:14,728 - DBG Calculated normalized_tez_am_container_size : 2048.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.0
2018-02-06 11:44:14,728 - DBG: Calculated and returning 'hive_tez_container_size' : 3072
2018-02-06 11:44:14,728 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0
2018-02-06 11:44:14,728 - DBG: Calculated 'slider_am_container_size' : 2048.0, using following: yarn_min_container_size : 2048.0
2018-02-06 11:44:14,728 - DBG: Calculated 'min_memory_required': 8192.0 using following : slider_am_container_size: 2048.0, normalized_tez_am_container_size : 2048.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 2048.0
2018-02-06 11:44:14,728 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 8192.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-06 11:44:14,728 - DBG: selected_queue_is_ambari_managed_llap = False
2018-02-06 11:44:14,728 - Entered __getSelectedQueueTotalCap fn() with llap_daemon_selected_queue_name= 'default'.
2018-02-06 11:44:14,728 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity
2018-02-06 11:44:14,729 - Queue list : ['root', 'default']
2018-02-06 11:44:14,729 - DBG: Selected queue name as: yarn.scheduler.capacity.root.capacity
2018-02-06 11:44:14,729 - Total capacity available for queue root is : 49152.0
2018-02-06 11:44:14,729 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity
2018-02-06 11:44:14,729 - Total capacity available for queue default is : 4915.2
2018-02-06 11:44:14,729 - DBG: Calculated 'default' queue available capacity : 4096.0, using following: llap_daemon_selected_queue_cap : 4915.2, yarn_min_container_size : 2048.0
2018-02-06 11:44:14,729 - DBG: Calculated 'num_llap_nodes_requested' : 1.0, using following: total_llap_mem_normalized : 4096.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-06 11:44:14,729 - Setting config 'num_llap_nodes' as : 1.0
2018-02-06 11:44:14,729 - Returning default AM percent value : '0.1' for queue : default
2018-02-06 11:44:14,730 - DBG: Calculated 'hive_tez_am_cap_available' : 409.6, using following: queue_am_fraction_perc : 0.1, total_llap_mem_normalized : 4096.0
2018-02-06 11:44:14,730 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 2048.0, using following : total_llap_mem_normalized : 4096.0, slider_am_container_size : 2048.0
2018-02-06 11:44:14,730 - Not enough capacity available on the cluster to run LLAP
2018-02-06 11:44:14,730 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration
2018-02-06 11:44:14,730 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 2048
2018-02-06 11:44:14,730 - Setting Yarn Repo user for Ranger.
2018-02-06 11:44:14,731 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,731 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,732 - DBG: Setting 'num_llap_nodes' config's READ ONLY attribute as 'True'.
2018-02-06 11:44:14,732 - Setting Hive Repo user for Ranger.
2018-02-06 11:44:14,733 - Not setting Kafka Repo user for Ranger.
2018-02-06 11:44:14,733 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-06 11:44:14,733 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,733 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,734 - DBG: Entered updateLlapConfigs
2018-02-06 11:44:14,734 - Is cluster create operation ? = False
2018-02-06 11:44:14,734 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-06 11:44:14,734 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.
2018-02-06 11:44:14,735 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,735 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,735 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.
2018-02-06 11:44:14,735 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'hunger_games', 'label': 'hunger_games'}, {'value': 'load_raw', 'label': 'load_raw'}, {'value': 'low_prior', 'label': 'low_prior'}, {'value': 'queries', 'label': 'queries'}]
2018-02-06 11:44:14,735 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,735 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,735 - YARN leaf Queues = set(['default', 'hunger_games', 'low_prior', 'load_raw', 'queries'])
2018-02-06 11:44:14,736 - DBG: llap_named_queue_selected_in_curr_invocation = False
2018-02-06 11:44:14,736 - DBG: Setting 'num_llap_nodes' config's READ ONLY attribute as 'True'.
2018-02-06 11:44:14,736 - Selected YARN queue for LLAP is : 'default'. Current YARN queues : ['default', 'hunger_games', 'low_prior', 'load_raw', 'queries']. Setting 'Number of LLAP nodes' visibility to 'False'.
2018-02-06 11:44:14,736 -
DBG: Performing LLAP config calculations ......
2018-02-06 11:44:14,736 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.0
2018-02-06 11:44:14,736 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 2048
2018-02-06 11:44:14,736 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 1024
2018-02-06 11:44:14,736 - DBG Calculated normalized_tez_am_container_size : 2048.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.0
2018-02-06 11:44:14,736 - DBG: Calculated and returning 'hive_tez_container_size' : 3072
2018-02-06 11:44:14,737 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0
2018-02-06 11:44:14,737 - DBG: Calculated 'slider_am_container_size' : 2048.0, using following: yarn_min_container_size : 2048.0
2018-02-06 11:44:14,737 - DBG: Calculated 'min_memory_required': 8192.0 using following : slider_am_container_size: 2048.0, normalized_tez_am_container_size : 2048.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 2048.0
2018-02-06 11:44:14,737 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 8192.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-06 11:44:14,737 - DBG: selected_queue_is_ambari_managed_llap = False
2018-02-06 11:44:14,737 - Entered __getSelectedQueueTotalCap fn() with llap_daemon_selected_queue_name= 'default'.
2018-02-06 11:44:14,737 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity
2018-02-06 11:44:14,737 - Queue list : ['root', 'default']
2018-02-06 11:44:14,737 - DBG: Selected queue name as: yarn.scheduler.capacity.root.capacity
2018-02-06 11:44:14,737 - Total capacity available for queue root is : 49152.0
2018-02-06 11:44:14,738 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity
2018-02-06 11:44:14,738 - Total capacity available for queue default is : 4915.2
2018-02-06 11:44:14,738 - DBG: Calculated 'default' queue available capacity : 4096.0, using following: llap_daemon_selected_queue_cap : 4915.2, yarn_min_container_size : 2048.0
2018-02-06 11:44:14,738 - DBG: Calculated 'num_llap_nodes_requested' : 1.0, using following: total_llap_mem_normalized : 4096.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-06 11:44:14,738 - Setting config 'num_llap_nodes' as : 1.0
2018-02-06 11:44:14,738 - Returning default AM percent value : '0.1' for queue : default
2018-02-06 11:44:14,738 - DBG: Calculated 'hive_tez_am_cap_available' : 409.6, using following: queue_am_fraction_perc : 0.1, total_llap_mem_normalized : 4096.0
2018-02-06 11:44:14,738 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 2048.0, using following : total_llap_mem_normalized : 4096.0, slider_am_container_size : 2048.0
2018-02-06 11:44:14,738 - Not enough capacity available on the cluster to run LLAP
2018-02-06 11:44:14,738 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration
2018-02-06 11:44:14,739 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 2048
2018-02-06 11:44:14,739 - Setting Yarn Repo user for Ranger.
2018-02-06 11:44:14,739 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,739 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,740 - No oozie configurations available
2018-02-06 11:44:14,742 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,742 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,742 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,742 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,743 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,744 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,745 - Updated 'tez-site' config 'tez.task.launch.cmd-opts' and 'tez.am.launch.cmd-opts' as : -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC{{heap_dump_opts}}
2018-02-06 11:44:14,745 - call type context : {'call_type': 'recommendConfigurationDependencies'}
2018-02-06 11:44:14,745 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,746 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,746 - DBG: Entered updateLlapConfigs
2018-02-06 11:44:14,746 - Is cluster create operation ? = False
2018-02-06 11:44:14,746 - Determining 'hive.llap.daemon.queue.name' config Property Attributes.
2018-02-06 11:44:14,746 - 'capacity-scheduler' not modified in the current Stack Advisor invocation.
2018-02-06 11:44:14,747 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,747 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,747 - 'capacity-scheduler' not changed in current Stack Advisor invocation. Retrieved the configs from services.
2018-02-06 11:44:14,747 - 'hive.llap.daemon.queue.name' config Property Attributes set to : [{'value': 'default', 'label': 'default'}, {'value': 'hunger_games', 'label': 'hunger_games'}, {'value': 'load_raw', 'label': 'load_raw'}, {'value': 'low_prior', 'label': 'low_prior'}, {'value': 'queries', 'label': 'queries'}]
2018-02-06 11:44:14,747 - 'capacity-scheduler' configs is passed-in as a single '\n' separated string. count(services['configurations']['capacity-scheduler']['properties']['capacity-scheduler']) = 77
2018-02-06 11:44:14,747 - Retrieved 'capacity-scheduler' received as dictionary : 'False'. configs : [('', ''), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.reporting.queries.priority', '0'), ('yarn.scheduler.capacity.root.accessible-node-labels', '*'), ('yarn.scheduler.capacity.maximum-am-resource-percent', '0.5'), ('yarn.scheduler.capacity.root.default.maximum-capacity', '20'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-applications', '10'), ('yarn.scheduler.capacity.root.production.batch_loading.queues', 'load_raw,low_prior'), ('yarn.scheduler.capacity.root.data_science.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.user-limit-factor', '2.5'), ('yarn.scheduler.capacity.root.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.user-limit-factor', '4.25'), ('yarn.scheduler.capacity.root.data_science.queues', 'hunger_games'), ('yarn.scheduler.capacity.root.data_science.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.queries.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.reporting.capacity', '20'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.default.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.ordering-policy', 'fifo'), ('yarn.scheduler.capacity.root.production.batch_loading.priority', '0'), ('yarn.scheduler.capacity.root.capacity', '100'), ('yarn.scheduler.capacity.root.production.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.user-limit-factor', '1.5'), ('yarn.scheduler.capacity.root.default.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.capacity', '25'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.reporting.queries.state', 'RUNNING'), ('yarn.scheduler.capacity.root.data_science.hunger_games.minimum-user-limit-percent', '25'), ('yarn.scheduler.capacity.root.reporting.queries.minimum-user-limit-percent', '33'), ('yarn.scheduler.capacity.root.data_science.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.priority', '0'), ('yarn.scheduler.capacity.root.reporting.priority', '0'), ('yarn.scheduler.capacity.root.data_science.hunger_games.state', 'RUNNING'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.state', 'RUNNING'), ('yarn.scheduler.capacity.root.reporting.queues', 'queries'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.priority', '0'), ('yarn.scheduler.capacity.root.reporting.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.acl_submit_applications', '*'), ('yarn.scheduler.capacity.node-locality-delay', '4'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.state', 'RUNNING'), ('yarn.scheduler.capacity.root.default.priority', '0'), ('yarn.scheduler.capacity.root.queues', 'data_science,default,production,reporting'), ('yarn.scheduler.capacity.maximum-applications', '10000'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.capacity', '100'), ('yarn.scheduler.capacity.root.reporting.queries.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.minimum-user-limit-percent', '100'), ('yarn.scheduler.capacity.root.acl_administer_queue', 'yarn'), ('yarn.scheduler.capacity.root.reporting.queries.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.capacity', '40'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.capacity', '75'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.capacity', '100'), ('yarn.scheduler.capacity.root.data_science.hunger_games.capacity', '100'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.acl_administer_queue', '*'), ('yarn.scheduler.capacity.root.data_science.hunger_games.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.data_science.capacity', '30'), ('yarn.scheduler.capacity.root.production.batch_loading.acl_submit_applications', '*'), ('yarn.scheduler.capacity.root.production.batch_loading.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.production.priority', '0'), ('yarn.scheduler.capacity.root.production.batch_loading.low_prior.ordering-policy', 'fair'), ('yarn.scheduler.capacity.root.production.maximum-capacity', '90'), ('yarn.scheduler.capacity.root.default.acl_submit_applications', 'yarn'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.maximum-capacity', '100'), ('yarn.scheduler.capacity.root.default.capacity', '10'), ('yarn.scheduler.capacity.root.data_science.maximum-capacity', '95'), ('yarn.scheduler.capacity.root.production.queues', 'batch_loading'), ('yarn.scheduler.capacity.root.production.batch_loading.load_raw.priority', '0'), ('yarn.scheduler.capacity.root.reporting.maximum-capacity', '80'), ('yarn.scheduler.capacity.root.data_science.hunger_games.acl_administer_queue', '*'), ('yarn.scheduler.capacity.queue-mappings-override.enable', 'false'), ('yarn.scheduler.capacity.root.reporting.acl_administer_queue', '*')]
2018-02-06 11:44:14,748 - YARN leaf Queues = set(['default', 'hunger_games', 'low_prior', 'load_raw', 'queries'])
2018-02-06 11:44:14,748 - DBG: llap_named_queue_selected_in_curr_invocation = False
2018-02-06 11:44:14,748 - DBG: Setting 'num_llap_nodes' config's READ ONLY attribute as 'True'.
2018-02-06 11:44:14,748 - Selected YARN queue for LLAP is : 'default'. Current YARN queues : ['default', 'hunger_games', 'low_prior', 'load_raw', 'queries']. Setting 'Number of LLAP nodes' visibility to 'False'.
2018-02-06 11:44:14,748 -
DBG: Performing LLAP config calculations ......
2018-02-06 11:44:14,748 - DBG: Calculated total_cluster_capacity : 49152.0, using following : node_manager_cnt : 2, yarn_nm_mem_in_mb : 24576.0
2018-02-06 11:44:14,748 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 2048
2018-02-06 11:44:14,748 - DBG: Calculated and returning 'tez_am_resource_memory_mb' as : 1024
2018-02-06 11:44:14,748 - DBG Calculated normalized_tez_am_container_size : 2048.0, using following : tez_am_container_size : 1024.0, total_cluster_capacity : 49152.0
2018-02-06 11:44:14,749 - DBG: Calculated and returning 'hive_tez_container_size' : 3072
2018-02-06 11:44:14,749 - DBG: Calculated mem_per_thread_for_llap : 3072.0, using following: yarn_nm_mem_in_mb_normalized : 24576.0, cpu_per_nm_host : 8.0
2018-02-06 11:44:14,749 - DBG: Calculated 'slider_am_container_size' : 2048.0, using following: yarn_min_container_size : 2048.0
2018-02-06 11:44:14,749 - DBG: Calculated 'min_memory_required': 8192.0 using following : slider_am_container_size: 2048.0, normalized_tez_am_container_size : 2048.0, mem_per_thread_for_llap : 3072.0, yarn_min_container_size : 2048.0
2018-02-06 11:44:14,749 - DBG: Calculated 'min_node_required': 1, using following : min_memory_required : 8192.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-06 11:44:14,749 - DBG: selected_queue_is_ambari_managed_llap = False
2018-02-06 11:44:14,749 - Entered __getSelectedQueueTotalCap fn() with llap_daemon_selected_queue_name= 'default'.
2018-02-06 11:44:14,749 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity
2018-02-06 11:44:14,749 - Queue list : ['root', 'default']
2018-02-06 11:44:14,749 - DBG: Selected queue name as: yarn.scheduler.capacity.root.capacity
2018-02-06 11:44:14,749 - Total capacity available for queue root is : 49152.0
2018-02-06 11:44:14,750 - DBG: Selected queue name as: yarn.scheduler.capacity.root.default.capacity
2018-02-06 11:44:14,750 - Total capacity available for queue default is : 4915.2
2018-02-06 11:44:14,750 - DBG: Calculated 'default' queue available capacity : 4096.0, using following: llap_daemon_selected_queue_cap : 4915.2, yarn_min_container_size : 2048.0
2018-02-06 11:44:14,750 - DBG: Calculated 'num_llap_nodes_requested' : 1.0, using following: total_llap_mem_normalized : 4096.0, yarn_nm_mem_in_mb_normalized : 24576.0
2018-02-06 11:44:14,750 - Setting config 'num_llap_nodes' as : 1.0
2018-02-06 11:44:14,750 - Returning default AM percent value : '0.1' for queue : default
2018-02-06 11:44:14,750 - DBG: Calculated 'hive_tez_am_cap_available' : 409.6, using following: queue_am_fraction_perc : 0.1, total_llap_mem_normalized : 4096.0
2018-02-06 11:44:14,750 - DBG: Calculated 'llap_mem_for_tezAm_and_daemons' : 2048.0, using following : total_llap_mem_normalized : 4096.0, slider_am_container_size : 2048.0
2018-02-06 11:44:14,750 - Not enough capacity available on the cluster to run LLAP
2018-02-06 11:44:14,750 - DBG: Something likely went wrong. recommendDefaultLlapConfiguration
2018-02-06 11:44:14,751 - DBG: 'yarn.scheduler.minimum-allocation-mb' read from services as : 2048
2018-02-06 11:44:14,751 - Setting Yarn Repo user for Ranger.
06 Feb 2018 11:44:14,820 INFO [ambari-client-thread-75953] StackAdvisorRunner:71 - advisor script stderr: