Support Questions

Find answers, ask questions, and share your expertise

I got this Error : Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) while submit workflow to oozie

Explorer

In my project, I was using the Oozie Client API. But for Kerberised authentication, I was using AuthOozieClient to authenticate a user to access Oozie, but I getting error

org.ietf.jgss.GSSException: Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:129) ~[na:1.8.0_144] at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95) ~[na:1.8.0_144] at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170) ~[na:1.8.0_144] at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:151) ~[na:1.8.0_144] at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:128) ~[na:1.8.0_144] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:297) ~[hadoop-auth-2.7.3.jar:na] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:287) ~[hadoop-auth-2.7.3.jar:na] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_144] at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_144] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:287) ~[hadoop-auth-2.7.3.jar:na] at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:205) ~[hadoop-auth-2.7.3.jar:na] at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215) ~[hadoop-auth-2.7.3.jar:na] at org.apache.oozie.client.AuthOozieClient.createConnection(AuthOozieClient.java:128) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient$1.doExecute(OozieClient.java:461) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.retry.ConnectionRetriableClient.execute(ConnectionRetriableClient.java:44) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.createRetryableConnection(OozieClient.java:458) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.getSupportedProtocolVersions(OozieClient.java:338) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.validateWSVersion(OozieClient.java:298) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.createURL(OozieClient.java:413) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.access$000(OozieClient.java:76) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:509) ~[oozie-client-4.1.0.jar:4.1.0] at org.apache.oozie.client.OozieClient.getCoordJobInfo(OozieClient.java:1246) ~[oozie-client-4.1.0.jar:4.1.0] at com.oi.hermes.scheduler.service.DashboardManagerService.getOozieCoordWorkflows(DashboardManagerService.java:67) ~[classes/:na] at com.oi.hermes.scheduler.controller.HermesJobController.getAllWorkFlowInfo(HermesJobController.java:106) ~[classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.8.RELEASE.jar:4.3.8.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) [tomcat-embed-core-8.5.14.jar:8.5.14] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.14.jar:8.5.14] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_144] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.14.jar:8.5.14] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]

Here is my Java code :

public List<CoordinatorExecutions> getOozieCoordWorkflows(String scheduleId) throws OozieClientException { InsertScheduleCoordinatorInfo scheduleCoordinatiorInfo = new InsertScheduleCoordinatorInfo(); scheduleCoordinatiorInfo = scheduleWorkflowInfoRepository.findOne(UUID.fromString(scheduleId)); String coordIdString = scheduleCoordinatiorInfo.getCoordinatorId(); AuthOozieClient wc=new AuthOozieClient(oozieUrl); CoordinatorJob coordinatorJob = wc.getCoordJobInfo(coordIdString); WorkflowJob workflowJob; List<CoordinatorAction> action = coordinatorJob.getActions(); CoordinatorExecutions coordinatorExecutions = null; List<CoordinatorExecutions> executionsList = new ArrayList<CoordinatorExecutions>(); for (CoordinatorAction coordinatorAction : action) { coordinatorExecutions = new CoordinatorExecutions(); coordinatorExecutions.setId(coordinatorAction.getId()); coordinatorExecutions.setStatus(coordinatorAction.getStatus().toString()); LOGGER.info(coordinatorAction.getExternalId()); workflowJob = wc.getJobInfo(coordinatorAction.getExternalId()); coordinatorExecutions.setCreatedTime(workflowJob.getStartTime().toString()); if (workflowJob.getEndTime() != null) { coordinatorExecutions.setLastModifiedTime(workflowJob.getEndTime().toString()); } else { coordinatorExecutions.setLastModifiedTime(""); } executionsList.add(coordinatorExecutions); } return executionsList; }

23 REPLIES 23

Expert Contributor

Hello @Dinesh Jadhav,

Could you please share few more details how you are running this project and what all system properties you are passing i.e kerberos configurations (Like conf, Principal name, login credential etc ) while executing project?

Explorer

I run my spring boot APP as a service on the cluster, And I don't know what are the Kerberos configurations we should give and where. I used the only AuthOozieClient wc=new AuthOozieClient(oozieClientUrl); class for authenticate user.

Explorer

Can you give me all the steps required for Kerberos configuration Over oozie

Expert Contributor

@Dinesh Jadhav

I can try to setup sample Spring boot application which submit oozie job and let you know. If you can attach your project code, It would be easier for me to check.

Explorer

can you give me your mail id?, I will mail you sample project because the file size is large

Explorer

Expert Contributor

Thanks for details. I will check and let you know.

Cloudera Employee

Hi @Dinesh Jadhav,

First, you should try using this constructor:

newAuthOozieClient("http://localhost:12000/oozie", "KERBEROS");

Also make sure that you ran kinit command before running Spring application.

Explorer

I tried that code but the job is not submitted.

Here is the stack trace:

2018-03-27 14:01:19.340 WARN 16410 --- [nio-8084-exec-6] o.a.h.h.s.DomainSocketFactory : The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]

Explorer

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]

Explorer

Actually, I want it to handle by the application. So I don't want to run kinit command.

so I added one class to project on GitHub. It will create a config file with given properties. and return back HTTP response with Authentication. So go through it and let me know its useful or not. if yes how?. https://github.com/dineshjadhav-oi/Oozie-User-Authentication

Explorer

I ran the kinit command before launch application but it will shows following error:

IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries = 1. Exception = Could not authenticate, GSSException: No valid credent ials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)

Mentor

@Dinesh Jadhav

Can you paste the content of the following file you can scramble the REALM and other sensitive info:

- kinit command 
- krb5.conf
- kdc.conf
- kadm5.acl

Make sure that you have a local copy of krb5.conf on all hosts and that the kadmin is up and running

Explorer

Sorry I cant provide you conf files. So if you send me to sample conf files then I can compare them with my config files.

Explorer

@Geoffrey Shelton Okot

Can you please send me sample configuration files.

Explorer

Hi @Geoffrey Shelton Okot

I attached a text file which has all config files here - kerberos-config.txt. So you can check it and let me know.

Mentor

@Dinesh Jadhav

There were a couple of errors in the kdc.conf,krb5.conf and kadm5.acl.

Please see the attached files. I would first ask you to backup your current kdc,krb5 and kadm5.acl files.

I have tried to separate the config files with --------

Please let me know if you have well understood if you need clarifications don't hesitate

Explorer
@Geoffrey Shelton Okot

I do the changes in conf files and run that code again, but it will give the same error. Can you give me working example?. Which connect to the oozie and get data from it with kerberos authentication. I think something is missing in the code, So I don't know what!

Mentor

@Dinesh Jadhav

Okay lets first eliminate the Kerberos error can you attach your current files below

  1. - krb5.conf
  2. - kdc.conf
  3. - kadm5.acl

Can you also run as root on the kdc server and see if you get any output

# kadmin.local 

Then look at this oozie config for kerberos

Explorer

I don't get you. what I need to run as root on kdc server?

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.