Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Mapreduce jobs are getting into pending state with zero resources

Mapreduce jobs are getting into pending state with zero resources

Explorer

Hi,

 

I have launched a single node cluster with OS as Centos 7.5, 4vcpu, 16GB RAM in azure. But when I create a table and insert some values into the table in hive its gets to pending state with zero resources while seeing in resource manager. And also when I open up resource manager port to all. Some unknown jobs by dr.who is getting created. 

No idea of what to do. Please help me with this 2 issue.

 

 

Thanks in advance.

1 REPLY 1
Highlighted

Re: Mapreduce jobs are getting into pending state with zero resources

Master Guru
For single node clusters, the default configuration is often set insufficiently to run any multi-container job. See https://community.cloudera.com/t5/Batch-Processing-and-Workflow/JOB-Stuck-in-Accepted-State/td-p/294... for raising this up.

For the second issue, "dr.who"-run jobs, this is more serious and is a result of your Azure instance's network access rules being permittive. These unidentified jobs are effectively run by hack-bots that discover your cluster is accessible and without security over the internet. See https://community.cloudera.com/t5/Batch-Processing-and-Workflow/What-is-Dr-who-user-100s-of-yarn-job... for prior discussion on this. Typically, limiting internet-external access to your cluster's ports will help cease this.