Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Cloudera Manager cannot find default kerberos realm

Labels:
avatar
author-rank orak
Explorer

Created ‎12-16-2018 12:13 AM

I've configured kerberos using cloudera manager 5.13 with open ldap as its backend and sssd for the groups name mapping. I'm able to successfully kinit and klist as well as run jobs on the cluster.

 

However, when I try to open the Snapshots section or the File browser section, I get the following exception:

 

 

com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Can't get Kerberos realm

         at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2263)

         at com.google.common.cache.LocalCache.get(LocalCache.java:4000)

         at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4789)

         at com.cloudera.cmf.service.GenericServiceCdhClient.<init>(GenericServiceCdhClient.java:148)

         at com.cloudera.cmf.service.GenericServiceCdhClient.<init>(GenericServiceCdhClient.java:102)

         at com.cloudera.cmf.service.hdfs.HdfsClient.<init>(HdfsClient.java:61)

         at com.cloudera.api.dao.impl.SnapshotManagerDaoImpl.getSnapshottableDirListing(SnapshotManagerDaoImpl.java:539)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:498)

         at com.cloudera.api.dao.impl.ManagerDaoBase.invokeMethodInExistingTransaction(ManagerDaoBase.java:327)

         at com.cloudera.api.dao.impl.ManagerDaoBase.invoke(ManagerDaoBase.java:274)

         at com.sun.proxy.$Proxy178.getSnapshottableDirListing(Unknown Source)

         at com.cloudera.server.web.cmf.bdr2.BDR2SnapshotPoliciesDTO.<init>(BDR2SnapshotPoliciesDTO.java:170)

         at com.cloudera.server.web.cmf.bdr2.BDR2SnapshotPoliciesDTO.<init>(BDR2SnapshotPoliciesDTO.java:134)

         at com.cloudera.server.web.cmf.bdr2.BDR2Controller.snapshotPoliciesJson(BDR2Controller.java:142)

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:498)

         at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)

         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436)

         at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424)

         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)

         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)

         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669)

         at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:574)

         at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)

         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)

         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)

         at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)

         at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:131)

         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)

         at com.jamonapi.http.JAMonServletFilter.doFilter(JAMonServletFilter.java:48)

         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)

         at com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter.doFilter(JavaMelodyFacade.java:109)

         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)

         at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)

         at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:125)

         at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

         at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)

         at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)

         at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)

         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)

         at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)

         at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)

         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)

         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)

         at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)

         at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)

         at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)

         at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)

         at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)

         at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)

         at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)

         at org.mortbay.jetty.handler.StatisticsHandler.handle(StatisticsHandler.java:53)

         at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)

         at org.mortbay.jetty.Server.handle(Server.java:326)

         at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)

         at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)

         at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)

         at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)

         at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)

         at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)

         at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Can't get Kerberos realm

         at com.google.common.base.Throwables.propagate(Throwables.java:160)

         at com.cloudera.cmf.cdhclient.CdhExecutorFactory.createExecutor(CdhExecutorFactory.java:294)

         at com.cloudera.cmf.cdhclient.CdhExecutorFactory.createExecutor(CdhExecutorFactory.java:417)

         at com.cloudera.cmf.service.GenericServiceCdhClient.newClient(GenericServiceCdhClient.java:289)

         at com.cloudera.cmf.service.GenericServiceCdhClient.access$100(GenericServiceCdhClient.java:56)

         at com.cloudera.cmf.service.GenericServiceCdhClient$2.call(GenericServiceCdhClient.java:144)

         at com.cloudera.cmf.service.GenericServiceCdhClient$2.call(GenericServiceCdhClient.java:135)

         at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4792)

         at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)

         at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)

         at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)

         at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)

         ... 87 more

Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Can't get Kerberos realm

         at java.util.concurrent.FutureTask.report(FutureTask.java:122)

         at java.util.concurrent.FutureTask.get(FutureTask.java:192)

         at com.cloudera.cmf.cdhclient.CdhExecutorFactory.createExecutor(CdhExecutorFactory.java:288)

         ... 97 more

Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm

         at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)

         at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:275)

         at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260)

         at org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:337)

         at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:331)

         at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:263)

         at com.cloudera.cmf.cdh5client.CDH5ObjectFactoryImpl.login(CDH5ObjectFactoryImpl.java:191)

         at com.cloudera.cmf.cdhclient.CdhExecutorFactory$SecureClassLoaderSetupTask.run(CdhExecutorFactory.java:579)

         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

         at java.util.concurrent.FutureTask.run(FutureTask.java:266)

         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

         at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.reflect.InvocationTargetException

         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

         at java.lang.reflect.Method.invoke(Method.java:498)

         at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84)

         at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)

         ... 12 more

Caused by: KrbException: Cannot locate default realm

         at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)

         ... 18 more

and the following is my effective krb.conf file generated by the cloduera manager:

 

[libdefaults]
default_realm = CLIENT.COM
dns_lookup_kdc = false
dns_lookup_realm = false
ticket_lifetime = 86400
renew_lifetime = 604800
forwardable = true
default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96
default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96
permitted_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96
udp_preference_limit = 1
kdc_timeout = 10000
default_realm = CLIENT.COM
dns_lookup_realm = false
dns_lookup_kdc = false
default_ccache_name = FILE:/tmp/krb5cc_%{uid}
 
[realms]
CLIENT.COM = {
kdc = d1master03-nn.client
admin_server = d1master03-nn.client
default_domain = .client
database_module = openldap_ldapconf
kdc=p1master03-nn.client
admin_server=p1master03-nn.client
}
[domain_realm]
.client = CLIENT.COM
client = CLIENT.COM
[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
 
[dbmodules]
openldap_ldapconf = {
db_library = kldap
ldap_kerberos_container_dn = cn=kerberos,dc=client,dc=com,dc=sa
ldap_kdc_dn = cn=Manager,dc=client,dc=com,dc=sa
# this object needs to have read rights on
# the realm container, principal container and realm sub-trees
ldap_kadmind_dn = cn=Manager,dc=client,dc=com,dc=sa
# this object needs to have read and write rights on
# the realm container, principal container and realm sub-trees
ldap_service_password_file = /etc/krb5.d/stash.keyfile
ldap_servers = ldapi:/// ldap:///d1master03-nn.client:389
ldap_conns_per_server = 5
}

Whats the issue here?

 

5,258 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank orak
Explorer

Created ‎12-19-2018 03:56 AM

I needed to restart the cloudera scm server by running the following command on the cluster where cloudera manager is installed:

 

systemctl restart cloudera-scm-server

systemctl restart cloudera-scm-agent

 

View solution in original post

5,206 Views
0 Kudos
1 REPLY 1

avatar
author-rank orak
Explorer

Created ‎12-19-2018 03:56 AM

I needed to restart the cloudera scm server by running the following command on the cluster where cloudera manager is installed:

 

systemctl restart cloudera-scm-server

systemctl restart cloudera-scm-agent

 

5,207 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements