Member since
11-09-2017
19
Posts
0
Kudos Received
0
Solutions
12-05-2017
10:06 PM
Dear @Aditya Sirna and @Jay Kumar SenSharma, Thank you for your valuable comments and help. I could resolve issues regarding to running Zeppelin HIVE into standalone Hadoop cluster. I have added following configurations to sort out the issue: 1- I have added hadoop.proxyuser.hiveuser.hosts=* and hadoop.proxyuser.hiveuser.groups=* into core-site.xml in Hadoop. 2- I copied HIVE JDBC jar file from a standalone cluster into Zeppelin JDBC interpreter: cp /usr/local/hive/lib/hive-jdbc-2.1.0.jar /usr/local/zeppelin/interpreter/jdbc/ 3- I ran /usr/local/hive/bin/hiveserver2 to be able to listen port 10000. 4- I configured dependencies in jdbc interpreter from Zeppelin Web UI as follows: org.apache.hive:hive-jdbc::2.1.0 (because I have installed Hive version 2.1.0) org.apache.hadoop:hadoop-common::2.8.1 (because I have installed Hadoop version 2.8.1)
... View more
12-05-2017
09:49 PM
@Jay Kumar SenSharma Thanks for comments. That is really helpful. Since I am using standalone hadoop not ambari, I configured core-site.xml with the same hadoop.proxyuser configuration as you specified. Now, Zeppelin is working with Hive JDBC.
... View more
12-05-2017
04:37 PM
@Aditya Sirna, Do you mean install Hive-server2?
... View more
12-05-2017
04:15 PM
@Aditya Sirna, In the standalone cluster: 1) Zeppelin Hive JDBC jar: /usr/local/zeppelin/interpreter/jdbc 2) Hive JDBC jar: /usr/local/hive/jdbc/hive-jdbc-2.1.0-standalone.jar I copied Hive JDBC into Zeppelin path. Should Hive be Hive-server2? Because I can run Hive Server2 which is a execution file in Hive folder. 3) Shall I change dependencis in jdbc interpreter setting as well? default is: org.apache.hive:hive-jdbc:0.14.0
... View more
12-05-2017
03:33 PM
Dear @Aditya Sirna, Could you please give me more hints? I do not know where are both exactly, I mean Zeppelin Hive JDBC and HIVE Server JDBC. tnx.
... View more
12-05-2017
02:14 PM
I am using Zeppelin and Hive in a single Hadoop cluster (Pseudo-distributed mode). I have edited jdbc in Zeppelin interpreter and I have added hive settings as suggested in https://zeppelin.apache.org/docs/0.7.3/interpreter/hive.html Now, I am executing the below commands in Zeppelin %jdbc(hive)
show databases; But, I am facing with the following error: java.sql.SQLException: Could not establish connection to jdbc:hive2://localhost:10000: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:14:13, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:336, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:279, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:189, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:414, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:310, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1377, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1362, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1152, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:622, java.lang.Thread:run:Thread.java:748, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:22:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:89, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:421, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1807, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy34:open::-1, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:327, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:29:7, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:578, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:513, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:165, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:User: root is not allowed to impersonate anonymous:54:25, org.apache.hadoop.ipc.Client:getRpcResponse:Client.java:1481, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1337, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:227, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:116, com.sun.proxy.$Proxy30:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:787, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:398, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeMethod:RetryInvocationHandler.java:163, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invoke:RetryInvocationHandler.java:155, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeOnce:RetryInvocationHandler.java:95, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:335, com.sun.proxy.$Proxy31:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:1700, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1436, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1436, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:674, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:622, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:550], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous), serverProtocolVersion:null)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:467)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:360)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:378)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:570)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:709)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.protocol.TProtocolException: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:14:13, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:336, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:279, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:189, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:414, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:310, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1377, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1362, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1152, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:622, java.lang.Thread:run:Thread.java:748, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:22:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:89, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:421, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1807, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy34:open::-1, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:327, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:29:7, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:578, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:513, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:165, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:User: root is not allowed to impersonate anonymous:54:25, org.apache.hadoop.ipc.Client:getRpcResponse:Client.java:1481, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1337, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:227, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:116, com.sun.proxy.$Proxy30:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:787, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:398, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeMethod:RetryInvocationHandler.java:163, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invoke:RetryInvocationHandler.java:155, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeOnce:RetryInvocationHandler.java:95, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:335, com.sun.proxy.$Proxy31:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:1700, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1436, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1436, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:674, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:622, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:550], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous), serverProtocolVersion:null)
at org.apache.hive.service.cli.thrift.TOpenSessionResp.validate(TOpenSessionResp.java:578)
at org.apache.hive.service.cli.thrift.TOpenSessionResp$TOpenSessionRespStandardScheme.read(TOpenSessionResp.java:676)
at org.apache.hive.service.cli.thrift.TOpenSessionResp$TOpenSessionRespStandardScheme.read(TOpenSessionResp.java:612)
at org.apache.hive.service.cli.thrift.TOpenSessionResp.read(TOpenSessionResp.java:520)
at org.apache.hive.service.cli.thrift.TCLIService$OpenSession_result$OpenSession_resultStandardScheme.read(TCLIService.java:2281)
at org.apache.hive.service.cli.thrift.TCLIService$OpenSession_result$OpenSession_resultStandardScheme.read(TCLIService.java:2266)
at org.apache.hive.service.cli.thrift.TCLIService$OpenSession_result.read(TCLIService.java:2213)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:156)
at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:143)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:456)
... 27 more
Please guide me in this matter.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Zeppelin
11-27-2017
10:14 PM
I have installed Ambari and I can use Hive View 2.0 for executing Hive Queries. Due to my hardware issues (less RAM), Ambari server is very slow sometimes. For this reason, I wana run Hive queries in a command shell not in Hive View in Ambari. Is it possible? If so, how is it possible? I would appreciate any help.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive
11-13-2017
12:47 AM
Thanks for link. One step is about Hive configuration, but I do not want it. Or is it mandatory for Pig View? this link: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-views/content/setup_WebHCat_proxy_user_pig_view.html
... View more
11-13-2017
12:31 AM
Dear Dinesh, HDP version is 2.6. Pig View Configuration is in the below figures:
... View more
11-12-2017
11:07 PM
I have created a PIG view in Ambari Server and cannot access it due to error "Service 'webhcat' check failed: RA080 Can't determine WebHCat hostname neither by associated cluster nor by webhcat.hostname property" I used also the below configuration:
a.Go to Services > HDFS > Configs b.Under the Advanced tab,
navigate to the Custom
core-site section c.Click Add Property with the following inputs:
hadoop.proxyuser.hcat.groups=*
hadoop.proxyuser.hcat.hosts=*
webhcat.proxyuser.root.groups=*
webhcat.proxyuser.root.hosts=*
But, I have still problem. How can I resolve it?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive