I got this Error : Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) while submit workflow to oozie

[#1]

In my project, I was using the Oozie Client API. But for Kerberised authentication, I was using AuthOozieClient to authenticate a user to access Oozie, but I getting error

org.ietf.jgss.GSSException: Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:129) ~[na:1.8.0_144] at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95) ~[na:1.8.0_144] at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:151) ~[na:1.8.0_144] at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:128) ~[na:1.8.0_144] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:297) ~[hadoop-auth-2.7.3.jar:na] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:287) ~[hadoop-auth-2.7.3.jar:na] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_144] at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_144] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:287) ~[hadoop-auth-2.7.3.jar:na] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:205) ~[hadoop-auth-2.7.3.jar:na] at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215) ~[hadoop-auth-2.7.3.jar:na] at org.apache.oozie.client.AuthOozieClient.createConnection(AuthOozieClient.java:128) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient$1.doExecute(OozieClient.java:461) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.retry.ConnectionRetriableClient.execute(ConnectionRetriableClient.java:44) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.createRetryableConnection(OozieClient.java:458) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.getSupportedProtocolVersions(OozieClient.java:338) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.validateWSVersion(OozieClient.java:298) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.createURL(OozieClient.java:413) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.access$000(OozieClient.java:76) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:509) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.getCoordJobInfo(OozieClient.java:1246) ~[oozie-client-4.1.0.jar:4.1.0] at com.oi.hermes.scheduler.service.DashboardManagerService.getOozieCoordWorkflows(DashboardManagerService.java:67) ~[classes/:na] at com.oi.hermes.scheduler.controller.HermesJobController.getAllWorkFlowInfo(HermesJobController.java:106) ~[classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.14.jar:8.5.14] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_144] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.14.jar:8.5.14] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]

Here is my Java code :

public List<CoordinatorExecutions> getOozieCoordWorkflows(String scheduleId) throws OozieClientException { InsertScheduleCoordinatorInfo scheduleCoordinatiorInfo = new InsertScheduleCoordinatorInfo(); scheduleCoordinatiorInfo = scheduleWorkflowInfoRepository.findOne(UUID.fromString(scheduleId)); String coordIdString = scheduleCoordinatiorInfo.getCoordinatorId(); AuthOozieClient wc=new AuthOozieClient(oozieUrl); CoordinatorJob coordinatorJob = wc.getCoordJobInfo(coordIdString); WorkflowJob workflowJob; List<CoordinatorAction> action = coordinatorJob.getActions(); CoordinatorExecutions coordinatorExecutions = null; List<CoordinatorExecutions> executionsList = new ArrayList<CoordinatorExecutions>(); for (CoordinatorAction coordinatorAction : action) { coordinatorExecutions = new CoordinatorExecutions(); coordinatorExecutions.setId(coordinatorAction.getId()); coordinatorExecutions.setStatus(coordinatorAction.getStatus().toString()); LOGGER.info(coordinatorAction.getExternalId()); workflowJob = wc.getJobInfo(coordinatorAction.getExternalId()); coordinatorExecutions.setCreatedTime(workflowJob.getStartTime().toString()); if (workflowJob.getEndTime() != null) { coordinatorExecutions.setLastModifiedTime(workflowJob.getEndTime().toString()); } else { coordinatorExecutions.setLastModifiedTime(""); } executionsList.add(coordinatorExecutions); } return executionsList; }

[#2]

Hello @Dinesh Jadhav,

Could you please share few more details how you are running this project and what all system properties you are passing i.e kerberos configurations (Like conf, Principal name, login credential etc ) while executing project?

[#3]

I run my spring boot APP as a service on the cluster, And I don't know what are the Kerberos configurations we should give and where. I used the only AuthOozieClient wc=new AuthOozieClient(oozieClientUrl); class for authenticate user.

[#4]

Can you give me all the steps required for Kerberos configuration Over oozie

[#5]

@Dinesh Jadhav

I can try to setup sample Spring boot application which submit oozie job and let you know. If you can attach your project code, It would be easier for me to check.

[#6]

can you give me your mail id?, I will mail you sample project because the file size is large

[#7]

Click here to see my code https://github.com/dineshjadhav-oi/Oozie-User-Authentication

[#8]

Thanks for details. I will check and let you know.

[#9]

Hi @Dinesh Jadhav,

First, you should try using this constructor:

newAuthOozieClient("http://localhost:12000/oozie", "KERBEROS");

Also make sure that you ran kinit command before running Spring application.

[#10]

I tried that code but the job is not submitted.

Here is the stack trace:

2018-03-27 14:01:19.340 WARN 16410 --- [nio-8084-exec-6] o.a.h.h.s.DomainSocketFactory : The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]