Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Jobs running slow after a added a datanode

Highlighted

Jobs running slow after a added a datanode

Expert Contributor

Most of the jobs are running slow after I added a datanode which previously had some hardware issues, now corrected. But everytime ( 2 times i added) i add it the cluster, the jobs are running slow. I checked I/O rate with another server and it looks good too.. Please advise.

Thanks.

5 REPLIES 5

Re: Jobs running slow after a added a datanode

Contributor

Have you confirmed if there are containers being run on this node(and non local reads) thats causing job to be slow? If thats the case I would recommend to install only 'datanode' process first and once cluster is balanced (maybe after day) add 'nodemanager' process to run containers on the node.

Re: Jobs running slow after a added a datanode

Expert Contributor
@Rahul Reddy

Yes, i have node manager running as well. And yes, seems like containers are taking longer time. Can you tell me in what way it will help when i balance my cluster and then add nodemanager to it?

Re: Jobs running slow after a added a datanode

Contributor

We want to avoid non-local reads of data as much as possible for best performance. Details here:

http://ercoppa.github.io/HadoopInternals/AnatomyMapReduceJob.html#maptask-launch

Re: Jobs running slow after a added a datanode

Expert Contributor

I will try that and let you know how it works. Thanks.

Re: Jobs running slow after a added a datanode

Expert Contributor
@Rahul Reddy

I did load balancing and then realized one of the disks has I/O error out of nowhere .... so that was the issue.

Don't have an account?
Coming from Hortonworks? Activate your account here