Skip to main content
fransyk
fransyk
Silver ⭐️
July 23, 2020
Solved

Install DataNode Role on NameNode/SNN machine

Hi all,

can I install an HDFS datanode role in machine with installed HDFS NameNode or into machine with SecondaryNameNode?

I use CDH 6.3.2

 

Thanks,

Francesco

paras
parasAnswer
Gold ⭐️⭐️⭐️
July 23, 2020

@fransyk 

 

Yes, you can have datanode and namenode role together on a host as long as the hardware requirements are met for the service to avoid any out of memory issues.

 

Please refer 

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.html#concept_fzz_dq4_gbb

 

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

View in thread

4 replies

paras
Gold ⭐️⭐️⭐️
parasAnswer
Gold ⭐️⭐️⭐️
July 23, 2020

@fransyk 

 

Yes, you can have datanode and namenode role together on a host as long as the hardware requirements are met for the service to avoid any out of memory issues.

 

Please refer 

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.html#concept_fzz_dq4_gbb

 

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Translate
fransyk
fransykAuthor
Silver ⭐️
July 23, 2020

Hi paras,

is this solution suitable for a production environment? I read some topic that do not recommend it (link below)

 

https://serverfault.com/questions/763797/hadoop-name-node-and-data-node-on-the-same-machine

https://www.researchgate.net/post/Can_a_Hadoop_name_node_also_double_duty_as_a_data_node

https://www.ibm.com/support/pages/can-biginsights-have-datanode-same-machine-namenode#:~:text=Yes%2C%20you%20can%20have%20a,in%20the%20Hadoop%20file%20system.

 

Thanks,

Francesco

 

Translate
paras
Gold ⭐️⭐️⭐️
paras
Gold ⭐️⭐️⭐️
July 23, 2020

@fransyk 

 

It is not compulsory to have both the roles on different hosts. 

Having said that, in production environments it is recommended to differentiate worker hosts and master hosts.

 

Please refer below document for recommended role allocations for different cluster sizes:

 

https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_host_allocations.html#concept_f43_j4y_dw

 

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

 

Translate
mridley
Gold ⭐️
mridley
Gold ⭐️
July 23, 2020

The data node question has been answered, but one tangental comment - you say you are using the Secondary Name Node service. You almost certainly do not want to be using that. You do not get any HA with the SNN. What you probably want is the Standby Namenode. In Cloudera Manager you can enable HA from the HDFS service actions and that will replace your Secondary Name Node with a Standby Name Node.

Translate
    Terms & ConditionsAccessibility statement