Member since
08-11-2018
7
Posts
1
Kudos Received
0
Solutions
08-12-2018
02:45 PM
Try this config. instead and let's see the logs please (make sure you replace spaces with new lines 🙂 ) yarn.scheduler.capacity.maximum-am-resource-percent=0.2
yarn.scheduler.capacity.maximum-applications=10000
yarn.scheduler.capacity.node-locality-delay=40
yarn.scheduler.capacity.root.accessible-node-labels=*
yarn.scheduler.capacity.root.acl_administer_queue=*
yarn.scheduler.capacity.root.capacity=100
yarn.scheduler.capacity.root.default.acl_submit_applications=*
yarn.scheduler.capacity.root.default.capacity=0
yarn.scheduler.capacity.root.default.maximum-capacity=0
yarn.scheduler.capacity.root.default.state=RUNNING
yarn.scheduler.capacity.root.default.user-limit-factor=2
yarn.scheduler.capacity.root.queues=default,hs2,longrun,sqoop,talend
yarn.scheduler.capacity.queue-mappings=u:talenduser:talend,u:dbenefi:longrun,u:sqoop:sqoop
yarn.scheduler.capacity.queue-mappings-override.enable=false
yarn.scheduler.capacity.root.default.maximum-am-resource-percent=1
yarn.scheduler.capacity.root.hs2.acl_submit_applications=*
yarn.scheduler.capacity.root.hs2.capacity=25
yarn.scheduler.capacity.root.hs2.maximum-am-resource-percent=0.4
yarn.scheduler.capacity.root.hs2.maximum-applications=500
yarn.scheduler.capacity.root.hs2.maximum-capacity=100 yarn.scheduler.capacity.root.hs2.ordering-policy=fair
yarn.scheduler.capacity.root.hs2.ordering-policy.fair.enable-size-based-weight=true
yarn.scheduler.capacity.root.hs2.state=RUNNING
yarn.scheduler.capacity.root.hs2.user-limit-factor=4
yarn.scheduler.capacity.root.longrun.acl_submit_applications=*
yarn.scheduler.capacity.root.longrun.capacity=15
yarn.scheduler.capacity.root.longrun.maximum-capacity=15
yarn.scheduler.capacity.root.longrun.state=RUNNING
yarn.scheduler.capacity.root.longrun.user-limit-factor=1
yarn.scheduler.capacity.root.sqoop.acl_submit_applications=*
yarn.scheduler.capacity.root.sqoop.capacity=10
yarn.scheduler.capacity.root.sqoop.maximum-am-resource-percent=1
yarn.scheduler.capacity.root.sqoop.maximum-capacity=100
yarn.scheduler.capacity.root.sqoop.state=RUNNING
yarn.scheduler.capacity.root.sqoop.user-limit-factor=10
yarn.scheduler.capacity.root.talend.acl_submit_applications=*
yarn.scheduler.capacity.root.talend.capacity=25
yarn.scheduler.capacity.root.talend.maximum-applications=500
yarn.scheduler.capacity.root.talend.maximum-capacity=100
yarn.scheduler.capacity.root.talend.ordering-policy=fair
yarn.scheduler.capacity.root.talend.ordering-policy.fair.enable-size-based-weight=true
yarn.scheduler.capacity.root.talend.state=RUNNING
yarn.scheduler.capacity.root.talend.user-limit-factor=4
... View more
08-11-2018
11:23 PM
Could you go to ambari -> yarn -> Advanced -> Scheduler -> Capacity Scheduler. and paste it here.
... View more
08-11-2018
03:37 AM
What is the value of amshare on leafqueue try to set it to -1.0f
... View more
08-11-2018
02:59 AM
Could you open RM UI and share a screen shot for the main page as first step ? also can you share a full log for RM active node as next step ?
... View more
08-11-2018
12:39 AM
what is the max, min container size you have in your YARN configuration ?
... View more
08-11-2018
12:33 AM
1 Kudo
Use Knox If you are going to authenticate using LDAP use Knox IP eg.(jdbc:hive2://<knox_host>:8443/;ssl=true;sslTrustStore=/var/lib/knox/data/security/keystores/gateway.jks;trustStorePassword=<master_secret>;transportMode=http;httpPath=gateway/default/hive) and if you are going to use simple auth. connect to hive jdbc directly (zookeeper or hive server IP).
... View more