Install DataNode Role on NameNode/SNN machine

[#1]

Hi all,

can I install an HDFS datanode role in machine with installed HDFS NameNode or into machine with SecondaryNameNode?

I use CDH 6.3.2

Thanks,

Francesco

[#2 · Accepted solution]

@fransyk 

Yes, you can have datanode and namenode role together on a host as long as the hardware requirements are met for the service to avoid any out of memory issues.

Please refer 

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.h...

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

[#3 · Accepted solution]

@fransyk 

Yes, you can have datanode and namenode role together on a host as long as the hardware requirements are met for the service to avoid any out of memory issues.

Please refer 

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.h...

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

[#4]

Hi paras,

is this solution suitable for a production environment? I read some topic that do not recommend it (link below)

https://serverfault.com/questions/763797/hadoop-name-node-and-data-node-on-the-same-machine

https://www.researchgate.net/post/Can_a_Hadoop_name_node_also_double_duty_as_a_data_node

https://www.ibm.com/support/pages/can-biginsights-have-datanode-same-machine-namenode#:~:text=Yes%2C...

Thanks,

Francesco

[#5 · Accepted solution]

@fransyk 

It is not compulsory to have both the roles on different hosts. 

Having said that, in production environments it is recommended to differentiate worker hosts and master hosts.

Please refer below document for recommended role allocations for different cluster sizes:

https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_host_allocations.html#concept...

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

[#6]

The data node question has been answered, but one tangental comment - you say you are using the Secondary Name Node service. You almost certainly do not want to be using that. You do not get any HA with the SNN. What you probably want is the Standby Namenode. In Cloudera Manager you can enable HA from the HDFS service actions and that will replace your Secondary Name Node with a Standby Name Node.