Support Questions

pacosoplas · ‎05-17-2018

I have this error after start hive llap cluster

Failing this attempt. Failing the application.
org.codehaus.jackson.JsonParseException: Unexpected character ('A' (code 65)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@7906578e; line: 1, column: 2]

SLF4J: Found binding in [jar:file:/usr/hdp/2.6.4.0-91/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value
WARN cli.LlapServiceDriver: Ignoring unknown llap server parameter: [hive.aux.jars.path]
WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value

Prepared /var/lib/ambari-agent/tmp/llap-slider2018-05-17_07-42-10/run.sh for running LLAP on Slider
2018-05-17 02:42:27,891 - Run file path: /var/lib/ambari-agent/tmp/llap-slider2018-05-17_07-42-10/run.sh
2018-05-17 02:42:27,891 - Execute['/var/lib/ambari-agent/tmp/llap-slider2018-05-17_07-42-10/run.sh'] {'logoutput': True, 'user': 'hive'}

WARN cli.LlapStatusServiceDriver: Failed to parse application diagnostics from Yarn Diagnostics - Application application_1526542647534_0001 failed 2 times in previous 300000 milliseconds due to AM Container for appattempt_1526542647534_0001_000002 exited with  exitCode: -1000
For more detailed output, check the application tracking page: http://xxxxxxxx:8088/cluster/app/application_1526542647534_0001 Then click on links to logs of each attempt.
Diagnostics: Not able to initialize app directories in any of the configured local directories for app application_1526542647534_0001
Failing this attempt. Failing the application.
WARN cli.LlapStatusServiceDriver: AppDiagnostics not available for YARN application report
ERROR types.ApplicationDiagnostics: Exception while parsing json : org.codehaus.jackson.JsonParseException: Unexpected character ('A' (code 65)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@3df978b9; line: 1, column: 2]
Application application_1526542647534_0001 failed 2 times in previous 300000 milliseconds due to AM Container for appattempt_1526542647534_0001_000002 exited with  exitCode: -1000
For more detailed output, check the application tracking page: http://xxxxxxx:8088/cluster/app/application_1526542647534_0001 Then click on links to logs of each attempt.
Diagnostics: Not able to initialize app directories in any of the configured local directories for app application_1526542647534_0001
Failing this attempt. Failing the application.
org.codehaus.jackson.JsonParseException: Unexpected character ('A' (code 65)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@3df978b9; line: 1, column: 2]
        at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:442) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.impl.ReaderBasedParser._handleUnexpectedValue(ReaderBasedParser.java:1198) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.impl.ReaderBasedParser.nextToken(ReaderBasedParser.java:485) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.map.ObjectMapper._initForReading(ObjectMapper.java:2770) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2718) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863) ~[hive-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.apache.slider.api.types.ApplicationDiagnostics.fromJson(ApplicationDiagnostics.java:132) [slider-core-0.92.0.2.6.4.0-91.jar:?]
        at org.apache.hadoop.hive.llap.cli.LlapSliderUtils.getApplicationDiagnosticsFromYarnDiagnostics(LlapSliderUtils.java:109) [hive-llap-server-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.apache.hadoop.hive.llap.cli.LlapStatusServiceDriver.processAppReport(LlapStatusServiceDriver.java:418) [hive-llap-server-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.apache.hadoop.hive.llap.cli.LlapStatusServiceDriver.run(LlapStatusServiceDriver.java:247) [hive-llap-server-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
        at org.apache.hadoop.hive.llap.cli.LlapStatusServiceDriver.main(LlapStatusServiceDriver.java:914) [hive-llap-server-2.1.0.2.6.4.0-9

WARN cli.LlapStatusServiceDriver: Failed to parse application diagnostics from Yarn Diagnostics - Application application_1526542647534_0001 failed 2 times in previous 300000 milliseconds due to AM Container for appattempt_1526542647534_0001_000002 exited with  exitCode: -1000
For more detailed output, check the application tracking page: http://lbkbd1.liberbank.cloud:8088/cluster/app/application_1526542647534_0001 Then click on links to logs of each attempt.
Diagnostics: Not able to initialize app directories in any of the configured local directories for app application_1526542647534_0001
Failing this attempt. Failing the application.
WARN cli.LlapStatusServiceDriver: AppDiagnostics not available for YARN application report
ERROR types.ApplicationDiagnostics: Exception while parsing json : org.codehaus.jackson.JsonParseException: Unexpected character ('A' (code 65)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@7906578e; line: 1, column: 2]
Application application_1526542647534_0001 failed 2 times in previous 300000 milliseconds due to AM Container for appattempt_1526542647534_0001_000002 exited with  exitCode: -1000
For more detailed output, check the application tracking page: http://lbkbd1.liberbank.cloud:8088/cluster/app/application_1526542647534_0001 Then click on links to logs of each attempt.
Diagnostics: Not able to initialize app directories in any of the configured local directories for app application_1526542647534_0001
Failing this attempt. Failing the application.
org.codehaus.jackson.JsonParseException: Unexpected character ('A' (code 65)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@7906578e; line: 1, column: 2]

Any suggestion??

yves_d · ‎06-07-2018

Hi,

I had the same issue and the solution came from HW's support team, as follow:

Try the following:-

Add -Dzookeeper.skipACL=yes under Zookeeper configs in Advance zookeeper-env

Just add it after JVM Flag for eg:-

export SERVER_JVMFLAGS="$SERVER_JVMFLAGS -Dzookeeper.skipACL=yes -Djava.security.auth.login.config={{zk_server_jaas_file}}"

Once done do a rolling restart of the Zookeeper server and the required services and try to start the LLAP from Ambari.

I hope that works for you too!

,

Hi,

I add the same issue and the solution came from the HW support team as follow:

Try the following:-

Add -Dzookeeper.skipACL=yes under Zookeeper configs in Advance zookeeper-env

Just add it after JVM Flag for eg:-

export SERVER_JVMFLAGS="$SERVER_JVMFLAGS -Dzookeeper.skipACL=yes -Djava.security.auth.login.config={{zk_server_jaas_file}}"

Once done do a rolling restart of the Zookeeper server and the required services and try to start the LLAP from Ambari.

Hope that works for you too!

eluppol · ‎01-11-2019

Hi,

I have the same issue with Hive LLAP.

I tried the solution above, but it didn't change anything. Hive LLAP still cannot start with the same error.

Any ideas why this is happening and how to fix it?

,

Hi,

I have the same issue with Hive LLAP start.

I tried the solution from above, but it didn't change anything. Hive LLAP still does not start with the same error.

Any ideas why it happens and how to fix it?

Support Questions

I cant start Hive LLAP Cluster