hi, On Ambari, i am trying to access the namenode UI through the KNOX portal using my credentials:   After entering my valid user and password i get the Knox portal again: knox logs: 21/02/04 18:25:48 ||aa72c68a-11yf-10ae-b720-c01b2b456pcq|audit|00.0.00.00|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|unavailable|Request method: POST 21/02/04 18:25:49 ||aa72c68a-11yf-10ae-b720-c01b2b456pcq|audit|00.0.00.00|KNOXSSO|user1|||authentication|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success| 21/02/04 18:25:49 ||aa72c68a-11yf-10ae-b720-c01b2b456pcq|audit|00.0.00.00|KNOXSSO|user1|||authentication|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success|Groups: [CN=HDP_DEV_ADMIN_HDP,OU=HDP,OU=Applications,OU=Groups,OU=Cal,DC=corp,DC=cal,DC=ca, CN=HDP_USERS,OU=HDP,OU=Applications,OU=Groups,OU=Cal,DC=corp,DC=cal,DC=ca, CN=HDP_PRD_CG_SS_STOR_ANY_RW,OU=HDP,OU=Applications,OU=Groups,OU=Cal,DC=corp,DC=cal,DC=ca, CN=HDP_USERS,OU=HDP,OU=Applications,OU=Groups,OU=Cal,DC=corp,DC=cal,DC=ca, CN=HDP_PRD_ADMIN_HDP,OU=HDP,OU=Applications,OU=Groups,OU=Cal,DC=corp,DC=cal,DC=ca] 21/02/04 18:25:49 |||audit|00.0.00.00|KNOXSSO|user1|||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success|Response status: 303 21/02/04 18:25:49 ||d111c112-1c11-1ec1-bc1b-1116039111dq|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/redirecting.html?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|unavailable|Request method: GET 21/02/04 18:25:49 ||d111c112-1c11-1ec1-bc1b-1116039111dq|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/redirecting.html?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success| 21/02/04 18:25:49 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/redirecting.html?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success|Response status: 200 21/02/04 18:25:49 ||d221c112-1c12-1ec1-bc1b-2212039121dq|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|unavailable|Request method: GET 21/02/04 18:25:49 ||d221c112-1c12-1ec1-bc1b-2212039121dq|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|success| 21/02/04 18:25:49 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/styles/bootstrap.min.css|success|Response status: 200 21/02/04 18:25:49 ||d331j332-4u44-3ec3-bc3b-3333034444dq|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/knox.css|unavailable|Request method: GET 21/02/04 18:25:49 ||d331j332-4u44-3ec3-bc3b-3333034444dq|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/styles/knox.css|success| 21/02/04 18:25:49 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/styles/knox.css|success|Response status: 200 21/02/04 18:25:50 ||11d9410d-8181-861c-7145-8418b134c177|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/images/loading.gif|unavailable|Request method: GET 21/02/04 18:25:50 ||11d9410d-8181-861c-7145-8418b134c177|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/images/loading.gif|success| 21/02/04 18:25:50 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/images/loading.gif|success|Response status: 200 21/02/04 18:25:50 ||5bb8b899-b412-4105-bd56-ab7i1ea7auab|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/redirecting.jsp?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|unavailable|Request method: GET 21/02/04 18:25:50 ||5bb8b899-b412-4105-bd56-ab7i1ea7auab|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/redirecting.jsp?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success| 21/02/04 18:25:50 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/redirecting.jsp?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success|Response status: 200 21/02/04 18:25:50 ||aeb1ce2a-1147-4d3a-9cd8-722623b9d349|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/images/loading.gif|unavailable|Request method: GET 21/02/04 18:25:50 ||aeb1ce2a-1147-4d3a-9cd8-722623b9d349|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/images/loading.gif|success| 21/02/04 18:25:50 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/images/loading.gif|success|Response status: 200 21/02/04 18:25:50 ||25cdb2ea-8dcf-44ae-919b-b7a5a58c26b7|audit|00.0.00.00|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|unavailable|Request method: GET 21/02/04 18:25:50 |||audit|00.0.00.00|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success|Response status: 401 21/02/04 18:25:50 ||5856cc3a-0fa2-4a1b-8429-921fcfb370b7|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/login.html?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|unavailable|Request method: GET 21/02/04 18:25:50 ||5856cc3a-0fa2-4a1b-8429-921fcfb370b7|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/login.html?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success| 21/02/04 18:25:50 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/login.html?originalUrl=https://Hdp-server-ms01.cal.com:50470/index.html|success|Response status: 200 21/02/04 18:25:50 ||42901bbb-i0f3-4964-b41e-a0d85bc3b247|audit|00.0.00.00|knoxauth||||access|uri|/gateway/knoxsso/knoxauth/styles/hwx-login.css|unavailable|Request method: GET 21/02/04 18:25:50 ||42901bbb-i0f3-4964-b41e-a0d85bc3b247|audit|00.0.00.00|knoxauth|anonymous|||authentication|uri|/gateway/knoxsso/knoxauth/styles/hwx-login.css|success| 21/02/04 18:25:50 |||audit|00.0.00.00|knoxauth|anonymous|||access|uri|/gateway/knoxsso/knoxauth/styles/hwx-login.css|success|Response status: 200 However If i enter the wrong credentials, i get the following message:   knox logs: ... View more

Hi, I have zeppelin with spark interpreter. I tried to run the following command in a shell interpreter: %spark2 import org.apache.spark.sql.SQLContext import org.apache.spark.sql.types._ import org.apache.spark.sql.functions._ import org.apache.spark.sql._ import spark.implicits._ import spark.sqlContext.implicits._ val textFile = spark.read.textFile("/user/user1/test/test.txt")     we get the following error message: java.lang.NullPointerException at org.apache.thrift.transport.TSocket.open(TSocket.java:209) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:62) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:133) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:165) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:132) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:299) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:407) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) java.lang.NullPointerException at org.apache.thrift.transport.TSocket.open(TSocket.java:209) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:62) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:133) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:165) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:132) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:299) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:407) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)   Any idea on how to fix this issue? ... View more

        HDP-3.0.1.0 Spark2 2.3.0 Zeppelin Notebook 0.8.0 Kerberos 1.10.3-30 Apache Hadoop Multi-Node Kerberized Cluster using a HA proxy. with the following lines in haproxy.cfg: listen spark_livy bind <HA_PROXY_IP>:8999 mode tcp option tcplog server spark_livy_1 <HOST1_URL>:8999 check server spark_livy_2 <HOST2_URL>:8999 check       HI, When we try to run a query in zeppelin using LIVY2 interpretor, we have the following message that pop up:         org.springframework.web.client.RestClientException: Error running rest call; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://<HA_PROXY_URL>:8999/sessions/768/statements": <HA_PROXY_URL>:8999 failed to respond; nested exception is org.apache.http.NoHttpResponseException: <HA_PROXY_URL>:8999 failed to respond           On the LIVY2 server logs (livy-livy-server.out) we have the following line that appears:         21/01/12 18:53:22 INFO InteractiveSession: Stopping InteractiveSession 768... 21/01/12 18:53:23 WARN RpcDispatcher: [ClientProtocol] Closing RPC channel with 1 outstanding RPCs. 21/01/12 18:53:23 INFO InteractiveSession: Stopped InteractiveSession 768. 21/01/13 10:28:47 WARN AuthenticationFilter: AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature           On the logs (zeppelin-interpreter-livy2-zeppelin-<host>.log) in the zeppelin host the following line appears:         ERROR [2021-01-13 12:01:48,906] ({pool-2-thread-8} LivySharedInterpreter.java[interpret]:83) - Fail to interpret:print('hello mister ') org.apache.zeppelin.livy.LivyException: org.springframework.web.client.RestClientException: Error running rest call; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://<HA_PROXY_URL>:8999/sessions/768/statements": <HA_PROXY_URL>:8999 failed to respond; nested exception is org.apache.http.NoHttpResponseException: <HA_PROXY_URL>:8999 failed to respond at org.apache.zeppelin.livy.BaseLivyInterpreter.callRestAPI(BaseLivyInterpreter.java:733) at org.apache.zeppelin.livy.BaseLivyInterpreter.executeStatement(BaseLivyInterpreter.java:581) at org.apache.zeppelin.livy.BaseLivyInterpreter.interpret(BaseLivyInterpreter.java:393) at org.apache.zeppelin.livy.LivySharedInterpreter.interpret(LivySharedInterpreter.java:81) at org.apache.zeppelin.livy.BaseLivyInterpreter.interpret(BaseLivyInterpreter.java:251) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:103) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:633) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.springframework.web.client.RestClientException: Error running rest call; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://<HA_PROXY_URL>:8999/sessions/768/statements": <HA_PROXY_URL>:8999 failed to respond; nested exception is org.apache.http.NoHttpResponseException: <HA_PROXY_URL>:8999 failed to respond at org.springframework.security.kerberos.client.KerberosRestTemplate.doExecute(KerberosRestTemplate.java:196) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498) at org.apache.zeppelin.livy.BaseLivyInterpreter.callRestAPI(BaseLivyInterpreter.java:703) ... 15 more Caused by: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://<HA_PROXY_URL>:8999/sessions/768/statements": <HA_PROXY_URL>:8999 failed to respond; nested exception is org.apache.http.NoHttpResponseException: <HA_PROXY_URL>:8999 failed to respond at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:633) at org.springframework.security.kerberos.client.KerberosRestTemplate.doExecuteSubject(KerberosRestTemplate.java:202) at org.springframework.security.kerberos.client.KerberosRestTemplate.access$100(KerberosRestTemplate.java:67) at org.springframework.security.kerberos.client.KerberosRestTemplate$1.run(KerberosRestTemplate.java:191) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.springframework.security.kerberos.client.KerberosRestTemplate.doExecute(KerberosRestTemplate.java:187) ... 18 more Caused by: org.apache.http.NoHttpResponseException: <HA_PROXY_URL>:8999 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:91) at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:619) ... 24 more         Any idea on how to solve this issue?   Thank you in advance for your help.   note exemple:  <HA_PROXY_IP>   --> xx.x.xx.xx <HA_PROXY_URL> --> myhaproxy.test.com <HOST1_URL> --> hdp-dev-ms01.test.com  <HOST1> --> hdp-dev-ms01  <HOST2_URL> --> hdp-dev-ms02.test.com  <HOST2> --> hdp-dev-ms02    ... View more

Hi, We have the following alert on ambari from the MapReduce History Server Web UI: We have been receiving alerts today from MapReduce History Server Web UI encountered connection failed or operation timeout. As checked, History Server application didn't stop or restarted during the time of the alert.   Below is the alert message we have: History Server Web UI Connection failed to https://<host>:19890 (Execution of 'curl --location-trusted -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/b691e974-6769-43c2-bba1-e486d6007948 -c /var/lib/ambari-agent/tmp/cookies/b691e974-6769-43c2-bba1-e486d6007948 -w '%{http_code}' https://<host>:19890 --connect-timeout 5 --max-time 7 -o /dev/null 1>/tmp/tmptSz439 2>/tmp/tmpb5DaZ0' returned 28. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 270 100 270 0 0 1522 0 --:--:-- --:--:-- --:--:-- 1522 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:06 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:07 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:08 --:--:-- 0 curl: (28) Operation timed out after 7827 milliseconds with 0 out of -1 bytes received 401) Cluster: <cluster1> Host: <host> Could you please help us find why this alert keeps happening?   Thank you! ... View more

Hi i would like to put the HIVE metastores in HA  using haproxy load balancing. In the file config haproxy.cfg we already put the Hive HiveServer2 in HA using the following lines: listen hs2_meta_env bind xx.x.xx.xx:9083 mode tcp option tcplog balance first server hs2_prd_meta_1 <node1_url>:9083 check server hs2_prd_meta_2 <node2_url>:9083 check It works for HiveServer2. How can we do the same this time for metastores that are installed on  node1_url et node2_url? Thank you for your time. ... View more

Hi, I get the following message when i run a query in zeppelin using the livy2 interpreter: Any idea how to fix this issue?    HDP 3.0.1 Ambari 2.7.1.0 Zeppelin Notebook 0.8.0 Spark2 2.3.0 ... View more

I get below error sometimes while running a query in zeppelin through JDBC connection:     java.sql.SQLException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection closed by remote host at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:334) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:265) at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291) at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:291) at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:718) at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:801) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:103) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:633) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection closed by remote host at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161) at org.apache.thrift.transport.TSaslTransport.flush(TSaslTransport.java:501) at org.apache.thrift.transport.TSaslClientTransport.flush(TSaslClientTransport.java:37) at org.apache.hadoop.hive.metastore.security.TFilterTransport.flush(TFilterTransport.java:77) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at org.apache.hive.service.rpc.thrift.TCLIService$Client.send_ExecuteStatement(TCLIService.java:239) at org.apache.hive.service.rpc.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:231) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1572) at com.sun.proxy.$Proxy8.ExecuteStatement(Unknown Source) at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:323) ... 16 more Caused by: java.net.SocketException: Connection closed by remote host at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1547) at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159) ... 30 more   How can we resolved this ?   note:  ... View more

Hi, We are currently using HDP 3.0.1 on a multi nodes environment. We are not able to start the Metrics Collector, and the following message appears on the logs  /var/log/ambari-metrics-collector/ambari-metrics-collector.log   FATAL Error starting AMSApplicationServer org.apache.ambari.metrics.core.timeline.MetricsSystemInitializationException: Error creating Metrics Metadata Schema in HBase using Phoenix. at org.apache.ambari.metrics.core.timeline.discovery.TimelineMetricMetadataManager.initializeMetadata(TimelineMetricMetadataManager.java:169) at org.apache.ambari.metrics.core.timeline.discovery.TimelineMetricMetadataManager.initializeMetadata(TimelineMetricMetadataManager.java:134) at org.apache.ambari.metrics.core.timeline.HBaseTimelineMetricsService.initializeSubsystem(HBaseTimelineMetricsService.java:121) at org.apache.ambari.metrics.core.timeline.HBaseTimelineMetricsService.serviceInit(HBaseTimelineMetricsService.java:102)   We did try to: Restart Ambari agents Restart Metrics Monitors (roll back) restart Metrics Collector (rool back) Restart Ambari metric Restart HBASE Try to follow the steps on https://community.cloudera.com/t5/Community-Articles/How-to-solve-Ambari-Metrics-corrupted-data/ta-p/245261 Each try was not able to fix the error    Is there a way to resolve this problem?   Thank you ... View more

Hi, I am getting the following alert on Ambari for the Yarn service:  There are 2 stale alerts from 2 host(s): node-m03.host.com [NodeManager Health Summary (6h 34m)], node-wn01.host.com [NodeManager Health (7h)]   However, i don't see any nodemanagers with the unhealthy status on ambari... And i did a check on the YARN service with success.   I am not sure what exactly i should be doing to correct this alert. Could you please help me troubleshoot this issue?   Thank you   HDP 3.0.1 YARN 3.1.0 Ambari 2.7.1 ... View more

Hi,  We are currenly using HDP 3.0.1 with ambari 2.7.1. I was trying to solve a connection issue on the zeppelin ui that give to some of the users a disconnected status while they are using the chrome browser...(curiously it seems to be working just fine with firefox ).   I am able to see on the zeppelin notebook logs the following message:  Exception in login: org.apache.shiro.authc.AuthenticationException: Authentication token of type [class org.apache.zeppelin.realm.jwt.JWTAuthenticationToken] could not be authenticated by any configured realms. Please ensure that at least one realm can authenticate these tokens.   Also on the knox logs i have the following error message not sure if it is linked: Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C090400, comment: AcceptSecurityContext error, data 52e, v1db1   Could you please help us.   Thank you ... View more

Hi, I am currently using HDP 3.0.1 with ambari 2.7.1. What is the best way to set up a automatic snapshot of important folder on HDFS in a multi node environment?   Thank you ... View more

Hi,  A large amount of users and groups were imported in ranger after a ldap usersync and now i need to find a way to delete more than 200 of users and groups. I know that ranger has a api... i would like to know how to remove a list of external groups or users using that api or maybe another method that allow me to do it faster?     Thank you ... View more

HI, We have a multi host hadoop environment  that use HDP-3.0.1 with Nifi 1.7.0. The environment is Kerberised and we use ranger for the security, it is also connected to the LDAP/AD server for the user and groups.     Whenever i try to connect to NIFI using a u ser_test i get the message :    The u ser_test is part of a group who has full access to nifi in ranger... -Policy Name: All-nifi-Resource -NIfi Resource Identifier: * -Allow Conditions:    -Select Group: group_test    -Permissions: Read, Write   It seems that the AD group is not used when trying to connect in nifi (If i put the username u ser_test in the Select User section in Ranger it has access to nifi...).    I have the following msg in the nifi-user.log: 2019-11-13 12:18:29,368 INFO [NiFi Web Server-65454] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=xx-xx-xx-xxXX.XxxXx.xx, O=XxxXx Inc., L=XXXX, ST=XXXX, C=XX 2019-11-13 12:18:33,097 INFO [NiFi Web Server-65454] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response. 2019-11-13 12:18:33,136 INFO [NiFi Web Server-65286] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response. 2019-11-13 12:18:33,151 INFO [NiFi Web Server-65436] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<Knox JWT token>) GET https://xx-xx-xx-xxXX.XxxXx.xx:xxxx/nifi-api/flow/current-user (source ip: XX.XXX.XXX.XXX) 2019-11-13 12:18:33,151 INFO [NiFi Web Server-65436] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for user_test 2019-11-13 12:12:32,122 INFO [NiFi Web Server-65436] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[user_test], groups[none] does not have permission to access the requested resource. Unable to view the user interface. Returning Forbidden response.   I am not sure why none appears and not group_test. Thank you in advance for your help on that subject. ... View more

Hi, I am currently using Hortonworks HDP 3.01 with ambari 2.7.1.0 but it seems that restarting all services can be a challenge because starting services/components must be done in a certain order to avoid timeouts and errors.... Is there a way to script the process? There is a section "Service Auto Start" in ambari but it doesn't seems that Ambari is smart enough to start services/componants while taking into consideration the dependencies... Could you please help?   Thank you   ... View more

Hi, Is there a way in NIFI to compress a simple file exemple.txt to a specific compresssed gzip file like archive.gzip or archive.gz? When the compressed file (archive.gzip or archive.gz) is decompressed  the filename should be exemple.txt and not archive.   Thank you     ... View more

Hello, I am having an issue running a query on my host and i getting the following message:     Any idea how to solve this issue? ... View more