Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Console logs not showing in Beeline

avatar
Explorer

Hello,

 

Console logs are not showing in beeline for one of the hiveserver2. But it is working fine with another hiveserver2.

 

Compared the beeline env variables from both HS2 servers and found a difference in below env variable.

env:CONSOLETYPE=vt

 

Hence I have tried to set "env:CONSOLETYPE=vt" through beeline, but it is throwing below error.

 

Appreciate your help on this.

============

0: jdbc:hive2://HS2-host:> set env:CONSOLETYPE=vt;
Getting log thread is interrupted, since query is done!
Error: Error while processing statement: null (state=,code=1)
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: null
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:239)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:225)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:244)
at org.apache.hive.beeline.Commands.executeInternal(Commands.java:902)
at org.apache.hive.beeline.Commands.execute(Commands.java:1089)
at org.apache.hive.beeline.Commands.sql(Commands.java:985)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1085)
at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:917)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:845)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: null
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:374)
at org.apache.hive.service.cli.operation.HiveCommandOperation.runInternal(HiveCommandOperation.java:116)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:316)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:424)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:401)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:258)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:503)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

=========================

12 REPLIES 12

avatar
Champion

Hi 

 

I assume you are usinig Hive 0.14 and higer .

Could you please check these configuration this will enable logs for Beeline Clients. 

 

hive.server2.logging.operation.enabled
hive.server2.logging.operation.log.location
hive.server2.logging.operation.verbose (Hive 0.14 to 1.1)
hive.server2.logging.operation.level
hive.server2.logging.operation.enabled
Default Value: true

When true, HiveServer2 will save operation logs and make them available for clients.

hive.server2.logging.operation.log.location
Default Value: ${java.io.tmpdir}/${user.name}/operation_logs

Top level directory where operation logs are stored if logging functionality is enabled.

hive.server2.logging.operation.verbose
Default Value: false

When true, HiveServer2 operation logs available for clients will be verbose.  


hive.server2.logging.operation.level
Default Value: EXECUTION

HiveServer2 operation logging mode available to clients to be set at session level.
For this to work, hive.server2.logging.operation.enabled should be set to true. 
The allowed values are:
NONE: Ignore any logging.
EXECUTION: Log completion of tasks.
PERFORMANCE: Execution + Performance logs.
VERBOSE: All logs.

Fruther you can refer the below links 

 

https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hi...

avatar
Explorer

Hi @csguna

 

Thank you for the reply. 

 

Hive version is 1.1 and the cluster is managed by Cloudera manager.

 

For both HS2 hosts, the property is set to below values:

hive.server2.logging.operation.enabled=true
hive.server2.logging.operation.log.location=/var/log/hive/operation_logs
hive.server2.logging.operation.verbose=undefined
hive.server2.logging.operation.level=execution

 

But one HS2 is generating console logs for the jobs execution and the other one is not generating.

 

I have tried setting below value and retried, but no luck.

hive.server2.logging.operation.verbose=true

 

 

 

avatar
New Contributor

use this conf in hive-site.xml

<property>
<name>hive.async.log.enabled</name>
<value>false</value>
</property>