Logout ambari's connected users on ambari-server restart

enirys — Thu, 22 Jul 2021 09:58:01 GMT

Our ambari webui is encountering some errors when ambari-server restart. When some users are connected and we restart the ambari-server, it doesn't logout connected users and 401 errors appears

This behaviour is very constraining because it generate lot of lines in ambari logs (Thread)

How to force ambari to log out all connected users when ambari-server restart ?

Re: Logout ambari's connected users on ambari-server restart

Raamar — Sat, 31 Jul 2021 09:42:37 GMT

@enirys

- Can you confirm the value you have set for user.inactivity.timeout.default, user.inactivity.timeout.role.readonly.default

- Can you share the complete 401 error users are experiencing?

- Can you confirm if you have spnego authentication enabled for ambari?

- Attach the below logs after 401 error occurs in ambari UI

1. ambari.properties file
2. ambari-audit.log
3. ambari-server.log

Re: Logout ambari's connected users on ambari-server restart

enirys — Mon, 02 Aug 2021 14:58:03 GMT

@Raamar

Yes, I'm using spnego authentication with user inactivity properties

user.inactivity.timeout.default=600 user.inactivity.timeout.role.readonly.default=300

My ambari is behind a loadbalancer (nginx), bellow the 401 error logs :

/var/log/nginx/access.log

"GET /gateway/default/ambari/api/v1/clusters/prod/requests?to=end&page_size=10&fields=Requests&_=1625812254413 HTTP/1.1" 401 51 "https://knox.26f5de01-5e40-4d8a-98bd-a4353b7bf5e3.datalake/gateway/default/ambari" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0" "-"

ambari.properties

agent.package.install.task.timeout=36000 agent.stack.retry.on_repo_unavailability=false agent.stack.retry.tries=5 agent.task.timeout=2000 agent.threadpool.size.max=25 ambari-server.user=root ambari.ldap.isConfigured=true ambari.post.user.creation.hook=/var/lib/ambari-server/resources/scripts/post-user-creation-hook.sh ambari.post.user.creation.hook.enabled=true ambari.python.wrap=ambari-python-wrap authentication.kerberos.auth_to_local.rules=DEFAULT authentication.kerberos.enabled=true authentication.kerberos.spnego.keytab.file=/etc/security/keytabs/spnego.service.keytab authentication.kerberos.spnego.principal=HTTP/<ambari_host_fqdn> authentication.kerberos.user.types=LDAP authentication.ldap.baseDn=cn=accounts,dc=<domain>,dc=<domain>,dc=<domain> authentication.ldap.bindAnonymously=false authentication.ldap.dnAttribute=dn authentication.ldap.groupMembershipAttr=member authentication.ldap.groupNamingAttr=cn authentication.ldap.groupObjectClass=posixGroup authentication.ldap.managerDn=uid=ldapbind,cn=sysaccounts,cn=etc,dc=<domain>,dc=<domain>,dc=<domain> authentication.ldap.managerPassword=/etc/ambari-server/conf/ldap-password.dat authentication.ldap.primaryUrl=<ipa_host_fqdn>:636 authentication.ldap.useSSL=true authentication.ldap.userObjectClass=posixAccount authentication.ldap.usernameAttribute=uid

ambari-audit.log

2021-07-05T19:48:23.518+0200, User(null), RemoteIp(xxx.xxx.xxx.xxx), Operation(User login), Roles( ), Status(Failed), Reason(Authentication required)

ambari-server.log

02 Jul 2021 18:43:52,514 INFO [ambari-client-thread-792188] AmbariAuthToLocalUserDetailsService:109 - Translated knox/<knox_gateway>@<REALM> to knox using auth-to-local rules during Kerberos authentication. 02 Jul 2021 18:43:52,515 WARN [ambari-client-thread-792188] AmbariAuthToLocalUserDetailsService:143 - Failed find user account for user with username of knox during Kerberos authentication. 02 Jul 2021 18:43:52,516 WARN [ambari-client-thread-792188] AmbariKerberosAuthenticationFilter:149 - Negotiate Header was invalid: Negotiate YIIDlAYGKwYBBQUCoIIDiDCCA4SgDTALBgkqhkiG9xIBAgKhBAMCAXaiggNrBIIDZ2CCA2MGCSqGSIb3EgECAgEAboIDUjCCA06gAwIBBaEDAgEOogcDBQAgAAAAo4IB/WGCAfkwggH1oAMCAQWhMxsxMjZGNURFMDEtNUU0MC00RDhBLTk4QkQtQTQzNTNCN0JGNUUzLkRBVEFMQUtFLk9WSKJPME2gAwIBAKFGMEQbBEhUVFAbPG92aC1lbm9kZTYuMjZmNWRlMDEtNWU0MC00ZDhhLTk4YmQtYTQzNTNiN2JmNWUzLmRhdGFsYWtlLm92aKOCAWYwggFioAMCARKhAwIBAqKCAVQEggFQbqKii/FFb+dRh+NYor9rVsacj0GwwyOV1KCZLEFkyM7VEhjK/wV8gkJOL9V0u99lkAIPGFCI6TzHenX6VtiZEd/MvKox5b4V0REwnpTuX5vS5lXCfXtbEOF2SbXmch1/U5RCjVQr8+vm/8AYmILIsqnFFmU9AgFFtEtZsH1NZpiSbosJs3oDJ1yINbzuzrxpCJJTu2Rcv2j34mN8+SkA42D6ZO6yP1iwlTkWji9VEDSOYLdzzhPMq0rmUCuuHfDYHX2Dx09xx4h9C6d28E3+o07LmUTUOBbnkiwnk57HZ+muxfD7T93PHOcfcJGPPKFWnUzTBg/ZrNsU3M36Y6UdqJ3kH/nvu6TNZDy+EbJhztlz3H1xNlMQ7D9np+HNHd72CvgFFzEbLCqkR9WUUL0R9WIno10V2wX8gBXPV31qY+WU59vQHfa9kkmytR8a60sYpIIBNjCCATKgAwIBEqKCASkEggEle7R2DYTiWwLkpH4BKwKGmoZfD7ZX0garqKPDQ7HDBDJIXUTce4UELJ84bTwn/DtpnUeQ0rZ70FedqHlHutB60NsswXvzoozLk+7Meeo3c310iWNIEd++dtEo0ZUc4sE8CA984VPYHHral67L/L4rQgb39ikYhxo9ieLMMHZ7Im6uzsnQzH0shKSHgWsroneqSvQXAuNM+08OAWuP0znVFAddSpPiQYlnsf9LJcjrn0hLFQwoFV48HA7/PyGDZ8Jotw/UMD/sbokOazOvzubvGdoyS6+8xJlJnf+D9WneoviqOFn4Po6B8WANQuF02QbPOPeE4XKXA4n6iruZmdKeR1ksx6OuEGygRSs1lmsgOzZvrjNLCay2ty4qtsMUUCjQz9V8c4A= org.springframework.security.core.userdetails.UsernameNotFoundException: Failed find user account for user with username of knox during Kerberos authentication. at org.apache.ambari.server.security.authentication.kerberos.AmbariAuthToLocalUserDetailsService.createUser(AmbariAuthToLocalUserDetailsService.java:144) at org.apache.ambari.server.security.authentication.kerberos.AmbariAuthToLocalUserDetailsService.loadUserByUsername(AmbariAuthToLocalUserDetailsService.java:110) at org.springframework.security.kerberos.authentication.KerberosServiceAuthenticationProvider.authenticate(KerberosServiceAuthenticationProvider.java:66) at org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:156) at org.springframework.security.kerberos.web.authentication.SpnegoAuthenticationProcessingFilter.doFilter(SpnegoAuthenticationProcessingFilter.java:145) at org.apache.ambari.server.security.authentication.kerberos.AmbariKerberosAuthenticationFilter.doFilter(AmbariKerberosAuthenticationFilter.java:167) at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:120) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201) at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:139) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745)

Re: Logout ambari's connected users on ambari-server restart

enirys — Thu, 12 Aug 2021 08:50:04 GMT

@Raamarany advise please ?

Re: Logout ambari's connected users on ambari-server restart

enirys — Thu, 12 Aug 2021 14:54:54 GMT

This issue occurs when kerberos authentication is enabled

There is bug issue opened in ambari jira https://issues.apache.org/jira/browse/AMBARI-25127

To fix my problem, i just disabled kerberos authentication

authentication.kerberos.enabled=false

question Re: Logout ambari's connected users on ambari-server restart in Support Questions

Logout ambari's connected users on ambari-server restart

Re: Logout ambari's connected users on ambari-server restart

Re: Logout ambari's connected users on ambari-server restart

Re: Logout ambari's connected users on ambari-server restart

Re: Logout ambari's connected users on ambari-server restart