Can not start LLAP

[#1]

Hi ,guys

I can not start llap, I hope the community can help me solve this problem.

I do not understand why this is?

how to solve this problem?

[root@s3 hive]# bin/hive --service llap --name llap_service --instances 2 --size 1g --loglevel INFO --args "-XX:MaxMetaspaceSize=128m -verbose:class -XX:+UseParNewGC -Xmn1g -XX:+UseConcMarkSweepGC -XX:-UseBiasedLocking -XX:+PerfDisableSharedMem" --cache 1g --executors 1 --iothreads 10 --slider-am-container-mb 1024
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/tez/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
INFO cli.LlapServiceDriver: LLAP service driver invoked with arguments=[-directory, /tmp/staging-slider-X7qu26, --name, llap_service, --instances, 2, --size, 1g, --loglevel, INFO, --args, -XX:MaxMetaspaceSize=128m -verbose:class -XX:+UseParNewGC -Xmn1g -XX:+UseConcMarkSweepGC -XX:-UseBiasedLocking -XX:+PerfDisableSharedMem, --cache, 1g, --executors, 1, --iothreads, 10, --slider-am-container-mb, 1024]
INFO conf.HiveConf: Found configuration file file:/opt/hive/conf/hive-site.xml
Failed: Unable to find required config file: hive-site.xml
java.lang.Exception: Unable to find required config file: hive-site.xml
 at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:181)
 at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:104)
INFO cli.LlapServiceDriver: LLAP service driver finished

If you use another way to start

[root@s3 llap]# bin/llapDaemon.sh start
starting llapdaemon, logging to /tmp/llapDaemonLogs/llap-daemon-root-s3.ff.com.log and /tmp/llapDaemonLogs/llap-daemon-root-s3.ff.com.out

cat /tmp/llapDaemonLogs/llap-daemon-root-s3.ff.com.out

+ exec /opt/java/bin/java -Dproc_llapdaemon -Xms4096m -Xmx4096m -server -Djava.net.preferIPv4Stack=true -XX:NewRatio=8 -XX:+UseNUMA -XX:+PrintGCDetails -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=100M -Xloggc:/tmp/llapDaemonLogs/gc.log -XX:+UseParallelGC -Dlog4j.configurationFile=llap-daemon-log4j2.properties -Dllap.daemon.log.dir=/tmp/llapDaemonLogs -Dllap.daemon.log.file=llap-daemon-root-s3.ff.com.log -Dllap.daemon.root.logger=console -Dllap.daemon.log.level=DEBUG -classpath '/opt/hive/conf:/opt/hive/scripts/llap/lib/*:/opt/hive/scripts/llap/lib/tez/*:/opt/hive/scripts/llap/lib/udfs/*:.:/opt/hadoop/share/hadoop/*:/opt/hive/lib/*:/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*:/opt/hive/lib:/opt/tez:opt/tez/lib' org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon
Error: Could not find or load main class org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon

[#2]

It may be a bug in the way the tool looks for config files. Can you try adding

/opt/hive/conf directory to class path in the script to see if that works as a workaround?

[#3]

Thanks for your reply, I've added / opt / hive / conf to the classpath. But still can not start. Do you mean this?

[hive@s3 hive]$ env | grep CLASSPATH
HADOOP_CLASSPATH=/opt/hadoop/share/hadoop/*:/opt/hive/lib/*:/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*
CLASSPATH=/opt/tez:/opt/tez/lib/*:/opt/hadoop/share/hadoop/*:/opt/hive/lib/*:/opt/hive/conf
[hive@s3 hive]$ /opt/hadoop/bin/hadoop org.apache.hadoop.hive.llap.cli.LlapServiceDriver -directory /tmp/staging-slider-7zdmW2 --name llap_service --instances 2 --size 1g --loglevel INFO --cache 1g --executors 1 --iothreads 2 --slider-am-container-mb 1024 --args '-XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/tez/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/01/25 10:37:05 INFO cli.LlapServiceDriver: LLAP service driver invoked with arguments=-directory
18/01/25 10:37:05 INFO conf.HiveConf: Found configuration file null
Failed: Unable to find required config file: hive-site.xml
java.lang.Exception: Unable to find required config file: hive-site.xml
	at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:181)
	at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:104)
18/01/25 10:37:06 INFO cli.LlapServiceDriver: LLAP service driver finished

Another way

bin/llapDaemon.sh start

+ exec /opt/java/bin/java -Dproc_llapdaemon -Xms4096m -Xmx4096m -server -Djava.net.preferIPv4Stack=true -XX:NewRatio=8 -XX:+UseNUMA -XX:+PrintGCDetails -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=4 -XX:GCLogFileSize=100M -Xloggc:/tmp/llapDaemonLogs/gc.log -XX:+UseParallelGC -Dlog4j.configurationFile=llap-daemon-log4j2.properties -Dllap.daemon.log.dir=/tmp/llapDaemonLogs -Dllap.daemon.log.file=llap-daemon-hive-s3.ff.com.log -Dllap.daemon.root.logger=console -Dllap.daemon.log.level=DEBUG -classpath '/opt/hive/conf:/opt/hive/scripts/llap/lib/*:/opt/hive/scripts/llap/lib/tez/*:/opt/hive/scripts/llap/lib/udfs/*:.:/opt/hadoop/share/hadoop/*:/opt/hive/lib/*:/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*:/opt/hive/lib:/opt/tez:opt/tez/lib' org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon
Error: Could not find or load main class org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon

[#4]

Very, very puzzled and surprised, I solved the problem.

I just moved hive-site.xml file to /opt/tez/conf directory, the problem is solved.

I do not understand why.

[#5]

I think there were some classpath problems in either case. Unfortunately the servicedriver currently only packages the config (hive-site.xml) for LLAP daemon if it's present in classpath. So, it found a config outside of the classpath for its own use, but didn't package it, initially. The 2nd error seems to indicate that LlapDaemon class, and the LLAP jar itself, is not present when the daemon is started... not sure how such package was created without this jar.

Copying to another directory probably added the config to classpath so servicedriver could find it.