Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cant get master ket kerberos principal for use as renewer

Cant get master ket kerberos principal for use as renewer

Explorer

[ram@ngs-poc1 ~]$ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 10
Number of Maps  = 10
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
15/08/25 14:37:56 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
15/08/25 14:37:56 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
15/08/25 14:37:56 WARN security.UserGroupInformation: PriviledgedActionException as:ram@TCSHYDNEXTGEN.COM (auth:KERBEROS) cause:java.io.IOException: Can't get Master Kerberos principal for use as renewer
java.io.IOException: Can't get Master Kerberos principal for use as renewer
    at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:133)
    at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
    at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
    at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:142)
    at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1306)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1303)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1303)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1324)
    at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
    at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

5 REPLIES 5

Re: Cant get master ket kerberos principal for use as renewer

Cloudera Employee

When you kinit and then run

klist -f

Do you see a R flag? Are your expiration time and renew until time same on your ticket? If so have you configured ticket renewal on the KDC side?

If you are using MIT KDC in your kdc.conf you'll need something like

max_renewable_life = 7d

Highlighted

Re: Cant get master ket kerberos principal for use as renewer

Master Guru
You may observe this error if your /etc/hadoop/conf/ directory does not
carry entire set of configs (particularly, its either pointing to just
HDFS, and is missing YARN+MR configs). The error suggests that it cannot
obtain a config relating to a server principal from the loaded XMLs.

Could you post the output of commands "ls -l /etc/hadoop/" and "cat
/etc/hadoop/conf/yarn-site.xml"?

If you use CM to manage this cluster, you can resolve the issue by adding
on ngs-poc1 host a YARN Gateway (CM -> YARN -> Instances -> Add Role
Instances -> Add Gateway to ngs-poc1) and doing a cluster-wide client
configuration redeploy [1]

[1] - https://www.youtube.com/watch?v=4S9H3wftM_0

Cant get master ket kerberos principal for use as renewer

Explorer

Hi Harsh

Thanks for the support.

 

we have another cluster where gatway is not configured. But all the servers in the cluster are configured 

node manager service and we have not encountered this problem.

 

With the above what I understood is:

 

node manager acts as kerberos renwer, is it correct?

 

 

Re: Cant get master ket kerberos principal for use as renewer

Master Guru
In secure clusters, the NodeManagers are made into YARN Gateways
automatically by CM (and they carry valid /etc/hadoop/conf/) because of how
the YARN services require access to the container-executor configs.

The client programs just require proper configs to run, not a local
NodeManager service.

Re: Cant get master ket kerberos principal for use as renewer

New Contributor

Resolved the issue by adding gateway role to the node.I am facing this error for hbase bulk load so, I have added hbase, yarn and hdfs gateway roles.