Hi guys,
I am not able to see YARN Cluster Utilization Report on CDP after kerberos (active directory) is enabled. Earlier it was working fine. Cloudera Manager showing warning :
Bad : Failed to run MapReduce job to aggregate YARN container usage metrics.
The service monitor logs showing error: IllegalArgumentException: Null real user
I have verified that MapReduce Job User specified in yarn configs is a linux user present on all cluster nodes and also have principal created on AD. Also, Container Usage Metrics are getting generated at /tmp/cmYarnContainerMetrics. But I can see that no mapreduce job output generated at Output Directory "/tmp/cmYarnContainerMetricsAggregate" after enabling kerberos. Any idea why?
Below is the complete stack trace:
2022-05-06 21:00:32,833 WARN com.cloudera.cmon.firehose.polling.yarn.YarnUsageAggregator: Failed to run MapReduce job for yarn
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Null real user
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.cloudera.cmon.firehose.polling.yarn.YarnUsageAggregator.doWork(YarnUsageAggregator.java:367)
at com.cloudera.cmon.firehose.polling.yarn.YarnUsageAggregator.doWork(YarnUsageAggregator.java:79)
at com.cloudera.cmon.firehose.polling.AbstractFileSystemClientTask.doWorkWithClientConfig(AbstractFileSystemClientTask.java:55)
at com.cloudera.cmon.firehose.polling.AbstractCdhWorkUsingClientConfigs.doWork(AbstractCdhWorkUsingClientConfigs.java:45)
at com.cloudera.cmon.firehose.polling.CdhTask$InstrumentedWork.doWork(CdhTask.java:229)
at com.cloudera.cmf.cdhclient.CdhExecutor$SecurityWrapper$1.run(CdhExecutor.java:189)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at com.cloudera.cmf.cdh7client.security.UserGroupInformationImpl.doAs(UserGroupInformationImpl.java:42)
at com.cloudera.cmf.cdhclient.CdhExecutor$SecurityWrapper.doWork(CdhExecutor.java:186)
at com.cloudera.cmf.cdhclient.CdhExecutor$1.call(CdhExecutor.java:125)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Null real user
at org.apache.hadoop.security.UserGroupInformation.createProxyUser(UserGroupInformation.java:1489)
at com.cloudera.cmf.cdh7client.security.UserGroupInformationImpl.createProxyUser(UserGroupInformationImpl.java:67)
at com.cloudera.cmf.cdh7client.CDH7ObjectFactoryImpl.createProxyUser(CDH7ObjectFactoryImpl.java:91)
at com.cloudera.cmf.cdhclient.util.ImpersonatingTaskWrapper.doWork(ImpersonatingTaskWrapper.java:95)
... 11 more
Thanks,
Az
