Created 10-18-2017 07:56 AM
Hi all,
I get an error when trying to start Llap, HiveServer2, see the output log below
Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 681, in <module> HiveServerInteractive().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 229, in restart_llap self._llap_start(env) File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 332, in _llap_start code, output, error = shell.checked_call(cmd, user=params.hive_user, quiet = True, stderr=subprocess.PIPE, logoutput=True) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/hive-server2-hive2/bin/hive --service llap --slider-am-container-mb 512 --size 13824m --cache 1536m --xmx 9830m --loglevel INFO --output /var/lib/ambari-agent/tmp/llap-slider2017-10-18_07-47-53 --slider-placement 0 --skiphadoopversion --skiphbasecp --instances 4 --logger query-routing --args " -XX:+AlwaysPreTouch -Xss512k -XX:+UseG1GC -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts -XX:InitiatingHeapOccupancyPercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200 -XX:MetaspaceSize=1024m"' returned 1. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hive2/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.0.3-8/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value WARN cli.LlapServiceDriver: Ignoring unknown llap server parameter: [hive.aux.jars.path] WARN cli.LlapServiceDriver: Java versions might not match : JAVA_HOME=[/usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.141-1.b16.el7_3.x86_64],process jre=[/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.el7_3.x86_64/jre] WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value Traceback (most recent call last): File "/usr/hdp/2.6.0.3-8/hive2/scripts/llap/slider/package.py", line 201, in <module> main(sys.argv[1:]) File "/usr/hdp/2.6.0.3-8/hive2/scripts/llap/slider/package.py", line 153, in main os.makedirs(output) File "/usr/lib64/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/var/lib/ambari-agent/tmp/llap-slider2017-10-18_07-47-53'
The permissions seem to be fine, ambari is run as non root user
[root@xxx scripts]# ls -ltr /var/lib/ambari-agent total 76 -rwxr-xr-x. 1 ambari ambari 2326 Apr 3 2017 upgrade_agent_configs.py -rwxr-xr-x. 1 ambari ambari 6012 Apr 3 2017 install-helper.sh -rwxr-xr-x. 1 ambari ambari 1345 Apr 3 2017 ambari-sudo.sh -rwxr-xr-x. 1 ambari ambari 1123 Apr 3 2017 ambari-env.sh drwxr-xr-x. 2 ambari ambari 6 Apr 3 2017 lib drwxr-xr-x. 2 ambari ambari 6 Apr 3 2017 keys -rwxr-xr-x. 1 ambari ambari 2235 Apr 3 2017 upgrade_agent_configs.pyo drwxr-xr-x. 2 ambari ambari 51 Aug 7 09:24 tools drwxr-xr-x. 9 ambari ambari 4096 Aug 7 13:15 cache drwxr-xr-x. 4 ambari ambari 27 Aug 7 13:51 cred drwxrwxr-x. 3 ambari ambari 4096 Oct 18 09:19 tmp drwxr-xr-x. 4 ambari ambari 24576 Oct 18 11:23 data [root@xxx scripts]# ls -ltr /var/lib/ambari-agent/tmp total 468 -r-xr-xr-x. 1 ambari ambari 1556 Aug 7 13:49 changeUid.sh -rw-r--r--. 1 ambari ambari 1931 Aug 7 13:51 DBConnectionVerification.jar -rw-r--r--. 1 ambari ambari 11902 Aug 7 13:55 CredentialUtil.jar -rw-r--r--. 1 ambari ambari 446067 Aug 7 13:55 postgresql-jdbc.jar -rwxr-xr-x. 1 ambari ambari 1026 Aug 7 13:55 start_hiveserver2_script drwxrwxrwt. 3 hdfs hadoop 4096 Oct 13 12:38 hadoop_java_io_tmpdir -rwxr-xr-x. 1 ambari ambari 1079 Oct 18 09:19 start_hiveserver2_interactive_script [root@xxx scripts]# cat /var/lib/ambari-agent/tmp/start_hiveserver2_ [root@xxx scripts]# cat /var/lib/ambari-agent/tmp/start_hiveserver2_interactive_script # # # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # # HIVE_SERVER2_INTERACTIVE_OPTS=" -hiveconf hive.log.file=hiveserver2Interactive.log -hiveconf hive.log.dir=$5" HIVE_INTERACTIVE_CONF_DIR=$4 /usr/hdp/current/hive-server2-hive2/bin/hiveserver2 -hiveconf hive.metastore.uris=" " ${HIVE_SERVER2_INTERACTIVE_OPTS} > $1 2> $2 & echo $!|cat>$3[root@xxx scripts]#
Does any one have any ideas of what I can look at please?
Created 10-26-2017 07:58 AM
Hive Interactive service is started with 'hive' user and the user needs to write privileges on the /var/lib/ambari-agent/tmp folder.
Below is my cluster slider folder details:
ls -ltr /var/lib/ambari-agent/tmp drwxr-xr-x2 hive hadoop 4096 Aug 21 03:25 llap-slider2017-08-21_03-24-29 drwxr-xr-x2 hive hadoop 4096 Oct4 09:09 llap-slider2017-10-04_09-08-48
Created 06-01-2018 07:38 PM
Hive server interactive starts up as hive user, so the hive user needs access to the ambari agent tmp directory.
so please provide permission as "chmod 777 /var/lib/ambari-agent/tmp"