Support Questions
Find answers, ask questions, and share your expertise

Mapreduce jobs are getting into pending state with zero resources

Explorer

Hi,

 

I have launched a single node cluster with OS as Centos 7.5, 4vcpu, 16GB RAM in azure. But when I create a table and insert some values into the table in hive its gets to pending state with zero resources while seeing in resource manager. And also when I open up resource manager port to all. Some unknown jobs by dr.who is getting created. 

No idea of what to do. Please help me with this 2 issue.

 

 

Thanks in advance.

1 REPLY 1

Master Guru
For single node clusters, the default configuration is often set insufficiently to run any multi-container job. See https://community.cloudera.com/t5/Batch-Processing-and-Workflow/JOB-Stuck-in-Accepted-State/td-p/294... for raising this up.

For the second issue, "dr.who"-run jobs, this is more serious and is a result of your Azure instance's network access rules being permittive. These unidentified jobs are effectively run by hack-bots that discover your cluster is accessible and without security over the internet. See https://community.cloudera.com/t5/Batch-Processing-and-Workflow/What-is-Dr-who-user-100s-of-yarn-job... for prior discussion on this. Typically, limiting internet-external access to your cluster's ports will help cease this.