Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

"hadoop queue" cmd exits with NullPointer exception

Highlighted

"hadoop queue" cmd exits with NullPointer exception

Guru

Hi,

 

I configured YARN to use the CapacityScheduler and wanted to check the queues and the acl's afterwards on the shell.

The RM-WebUI shows the configured queues (root, emergency, lowprio) as expected, but if I execute the shell command

 

sudo -u yarn hadoop queue -list

 

I get the following exception:

 

DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

14/05/05 12:11:11 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/05/05 12:11:11 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapred.JobClient.getJobQueueInfoArray(JobClient.java:1045)
at org.apache.hadoop.mapred.JobClient.access$1800(JobClient.java:139)
at org.apache.hadoop.mapred.JobClient$10.run(JobClient.java:1063)
at org.apache.hadoop.mapred.JobClient$10.run(JobClient.java:1061)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.JobClient.getRootQueues(JobClient.java:1061)
at org.apache.hadoop.mapred.JobQueueClient.displayQueueList(JobQueueClient.java:148)
at org.apache.hadoop.mapred.JobQueueClient.run(JobQueueClient.java:101)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobQueueClient.main(JobQueueClient.java:240)

 

It doesn't matter if I execute the cmd as user 'yarn' or another one, the exception is the same. As mentioned earlier, the queues are looking fine in the ResourceManager-WebUI.

 

What's the problem on the shell execution ?!?!

1 REPLY 1

Re: "hadoop queue" cmd exits with NullPointer exception

Master Guru
This will happen if your host lacks client configurations, which causes the default mapreduce.framework.name value of "local" to be used instead of a connection to your actual RM.

Here's how you can manually reproduce it even if you have a working command already:

➜ ~ hadoop queue -Dmapreduce.framework.name=local -list
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapred.JobClient.getJobQueueInfoArray(JobClient.java:1045)
at org.apache.hadoop.mapred.JobClient.access$1800(JobClient.java:139)
at org.apache.hadoop.mapred.JobClient$10.run(JobClient.java:1063)
at org.apache.hadoop.mapred.JobClient$10.run(JobClient.java:1061)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.JobClient.getRootQueues(JobClient.java:1061)
at org.apache.hadoop.mapred.JobQueueClient.displayQueueList(JobQueueClient.java:148)
at org.apache.hadoop.mapred.JobQueueClient.run(JobQueueClient.java:101)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobQueueClient.main(JobQueueClient.java:240)

If you have proper client configs including mapred-site.xml and yarn-site.xml that correctly define the cluster information, then the command will work properly and list queues:

➜ ~ hadoop queue -list
14/07/20 18:15:24 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8032
======================
Queue Name : root.default
Queue State : running
Scheduling Info : Capacity: 0.0, MaximumCapacity: UNDEFINED, CurrentCapacity: 0.0