We have set up a 3 node cluster using HDP 2.6. While trying to restart the Hive service after enabling LLAP, we are getting the following error:-
200518568_0007/container_e22_1502200518568_0007_01_000002/tmp/ -Dlog4j.configurationFile=llap-daemon-log4j2.properties -Dllap.daemon.log.dir=/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/ -Dllap.daemon.log.file=llap-daemon-hive-hdmaster.icreate.bi.log'
+ LLAP_DAEMON_OPTS=' -Dhttp.maxConnections=3 -XX:+AlwaysPreTouch -Xss512k -XX:+UseG1GC -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts -XX:InitiatingHeapOccupancyPercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200 -XX:MetaspaceSize=1024m -server -Djava.net.preferIPv4Stack=true -XX:+UseNUMA -XX:+PrintGCDetails -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=100M -XX:+PrintGCDateStamps -Xloggc:/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002//gc.log -Djava.io.tmpdir=/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/tmp/ -Dlog4j.configurationFile=llap-daemon-log4j2.properties -Dllap.daemon.log.dir=/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/ -Dllap.daemon.log.file=llap-daemon-hive-hdmaster.icreate.bi.log -Dllap.daemon.root.logger=query-routing'
+ LLAP_DAEMON_OPTS=' -Dhttp.maxConnections=3 -XX:+AlwaysPreTouch -Xss512k -XX:+UseG1GC -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts -XX:InitiatingHeapOccupancyPercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200 -XX:MetaspaceSize=1024m -server -Djava.net.preferIPv4Stack=true -XX:+UseNUMA -XX:+PrintGCDetails -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=100M -XX:+PrintGCDateStamps -Xloggc:/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002//gc.log -Djava.io.tmpdir=/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/tmp/ -Dlog4j.configurationFile=llap-daemon-log4j2.properties -Dllap.daemon.log.dir=/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/ -Dllap.daemon.log.file=llap-daemon-hive-hdmaster.icreate.bi.log -Dllap.daemon.root.logger=query-routing -Dllap.daemon.log.level=INFO'
+ exec /usr/jdk64/jdk1.8.0_112/bin/java -Dproc_llapdaemon -Xms0m -Xmx0m -Dhttp.maxConnections=3 -XX:+AlwaysPreTouch -Xss512k -XX:+UseG1GC -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts -XX:InitiatingHeapOccupancyPercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200 -XX:MetaspaceSize=1024m -server -Djava.net.preferIPv4Stack=true -XX:+UseNUMA -XX:+PrintGCDetails -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=100M -XX:+PrintGCDateStamps -Xloggc:/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002//gc.log -Djava.io.tmpdir=/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/tmp/ -Dlog4j.configurationFile=llap-daemon-log4j2.properties -Dllap.daemon.log.dir=/u01/hadoop/yarn/log/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/ -Dllap.daemon.log.file=llap-daemon-hive-hdmaster.icreate.bi.log -Dllap.daemon.root.logger=query-routing -Dllap.daemon.log.level=INFO -classpath '/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/app/install//conf/:/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/app/install//lib/*:/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/app/install//lib/tez/*:/hadoop/yarn/local/usercache/hive/appcache/application_1502200518568_0007/container_e22_1502200518568_0007_01_000002/app/install//lib/udfs/*:.:' org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon
Invalid maximum heap size: -Xmx0m
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Node configurations are as follows:-
#Cores - 2
Memory - 16GB
yarn.scheduler.minimum-allocation-mb = 4608MB
hive.tez.container.size = 9216MB
tez.container.max.java.heap.fraction = 0.8
We have even tried providing the Xmx and Xms manually in hive.tez.java.opts to -Xmx7372m -Xms3686m. But it still seems to be taking the Xmx value as 0.
Can you please advise any property that we might have not setup properly?
