Reply
Expert Contributor
Posts: 96
Registered: ‎10-04-2017

Multiple datanodes on the same machine?

Hi,

 

Is it possible to have multiple datanodes on the same machine in Cloudera using CM? Obviously this can be done in plain Apache Hadoop. 

Posts: 519
Topics: 14
Kudos: 90
Solutions: 45
Registered: ‎09-02-2016

Re: Multiple datanodes on the same machine?

@RajeshBodolla

 

Not sure I get your intension to have multiple datanodes on the same machine

 

if you want to store data nodes in different/multiple directories in the same machine then you can use CM -> HDFS -> Configuration -> datanode.data.dir and specify your directories

Expert Contributor
Posts: 96
Registered: ‎10-04-2017

Re: Multiple datanodes on the same machine?

Hi,

 

I can add/remove the data directories. Want i want to add 2 new datanode instances on a single host from CM.

Posts: 224
Topics: 0
Kudos: 16
Solutions: 8
Registered: ‎08-19-2013

Re: Multiple datanodes on the same machine?

Hadoop wasn't designed to run multiple DataNodes on a single host and is prohibited by Cloudera Manager.   

 

The reason for a single DataNode per host is to prevent data loss.  Using the default replication factor of 3, every block in a file will be replicated to 3 different hosts.   If a host containing a block replica were to go down, the NameNode will mark the block as under-replicated.  A new copy of the block will be created on another DataNode bringing the number of replicas back to 3.  

 

If you do not care about data integrity my suggestion is to set the replication factor to 1 or use virtual hosts. 



David Enloe, Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Highlighted
Expert Contributor
Posts: 96
Registered: ‎10-04-2017

Re: Multiple datanodes on the same machine?

@denole

Yes, i agree with your point. my notion of this was to test when we have quickstart VM for testing.
Announcements