@Shelton  Thank you for your response but as of now first i am testing between server to server authentication and this is how my JAAS files looks like which i refereed from this wiki    https://cwiki.apache.org/confluence/display/ZOOKEEPER/Server-Server+mutual+authentication     QuorumServer { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="<keytab path>" storeKey=true useTicketCache=false debug=false principal="<principal>/_HOST@[REALM]"; }; QuorumLearner { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="<keytab path>" storeKey=true useTicketCache=false debug=false principal="<principal>/_HOST@[REALM]"; };       ... View more

@Shelton not able to execute any of the HDFS command due to kerberos.    hadoop fs -ls / 19/10/15 13:12:55 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/10/15 13:12:55 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 19/10/15 13:12:55 INFO retry.RetryInvocationHandler: java.io.IOException: DestHost:destPort hostname:8020 , LocalHost:localPort hostname/10.49.70.18:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS], while invoking ClientNamenodeProtocolTranslatorPB.getFileInfo over hostname10.49.70.14:8020 after 1 failover attempts. Trying to failover after sleeping for 1171ms ... View more

Thanks for your help Shelton.  I have one other question... trying to enable Kerberos in other server but it is saying not reachable.  Using similar configuration... server name different.  What would be be the issue?  btw. it is on HDP 3.1(Ambari 2.7) on CentOS 7 server.  What could be the reason to not able to reach the KDC? ... View more

PFA the below error logs : 19/10/09 16:09:32 DEBUG ServletHandler: chain=org.apache.hadoop.security.authentication.server.AuthenticationFilter-418c020b->org.apache.spark.ui.JettyUtils$$anon$3-75e710b@986efce7==org.apache.spark.ui.JettyUtils$$anon$3,jsp=null,order=-1,inst=true 19/10/09 16:09:32 DEBUG ServletHandler: call filter org.apache.hadoop.security.authentication.server.AuthenticationFilter-418c020b 19/10/09 16:09:32 DEBUG AuthenticationFilter: Got token null from httpRequest http://ip-10-0-10.184. ************:18081/ 19/10/09 16:09:32 DEBUG AuthenticationFilter: Request [http://ip-10-0-10-184.*****:18081/] triggering authentication. handler: class org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler 19/10/09 16:09:32 DEBUG AuthenticationFilter: Authentication exception: java.lang.IllegalArgumentException org.apache.hadoop.security.authentication.client.AuthenticationException: java.lang.IllegalArgumentException at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.authenticate(KerberosAuthenticationHandler.java:306) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:536) at org.spark_project.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493) at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.spark_project.jetty.server.Server.handle(Server.java:539) at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:333) at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.spark_project.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.spark_project.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.spark_project.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.spark_project.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalArgumentException at java.nio.Buffer.limit(Buffer.java:275) at org.apache.hadoop.security.authentication.util.KerberosUtil$DER.<init>(KerberosUtil.java:365) at org.apache.hadoop.security.authentication.util.KerberosUtil$DER.<init>(KerberosUtil.java:358) at org.apache.hadoop.security.authentication.util.KerberosUtil.getTokenServerName(KerberosUtil.java:291) at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.authenticate(KerberosAuthenticationHandler.java:285) ... 22 more 19/10/09 16:09:32 DEBUG GzipHttpOutputInterceptor: org.spark_project.jetty.server.handler.gzip.GzipHttpOutputInterceptor@17d4d832 exclude by status 403 19/10/09 16:09:32 DEBUG HttpChannel: sendResponse info=null content=HeapByteBuffer@26ea8849[p=0,l=365,c=32768,r=365]={<<<<html>\n<head>\n<me.../body>\n</html>\n>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=true committing=true callback=Blocker@137652aa{null} 19/10/09 16:09:32 DEBUG HttpChannel: COMMIT for / on HttpChannelOverHttp@4d71d816{r=2,c=true,a=DISPATCHED,uri=//ip-10-0-10-184.******:18081/} 403 java.lang.IllegalArgumentException HTTP/1.1 Date: Wed, 09 Oct 2019 16:09:32 GMT Set-Cookie: hadoop.auth=; HttpOnly Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html;charset=iso-8859-1   ... View more

Ok. WIll do. Thank you! ... View more

I'm in a test environment and it would really help if I'm already logged in as root to do everything I need, including accessing Hbase. I performed the changes and everything works perfectly! Thanks!  ... View more

Thank you very much! Your answer helped me find a way to solve the problem. I did not have Cloudera Manager running due to a lack of resources allocated to the virtual machine. After starting, I began to check warnings and configuration errors, and problems began to disappear one by one. Now I understand how to identify problems and solve them (this is not in the tutorial). Thanks again! ... View more

Thanks for the solution. ... View more