Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

can we deploy the HttpFS role on more than one node? is it best practice?

avatar
Explorer

can we deploy the HttpFS role on more than one node? is it best practice?

 

we had a problem with one of the namenodes (on node02) in our cluster and it was down for almost 20 hours.
once node02  was up, the active namenode ,on node01, started to alert that its not syncronized
after restart of the namenode on node02 the alert stopped
now active namenode is on node01
standby namenode is on node02

we can see that new logs are created on opt/hadoop/dfs/nn/current  on the actine namenode on node01
but no new files . on the standby namenode no node02 - is it OK ??

1 ACCEPTED SOLUTION

avatar
Mentor
> can we deploy the HttpFS role on more than one node? is it best practice?

Yes, the HttpFs service is an end-point for REST API access to HDFS, so you can deploy multiple instances and also consider load balancing (might need sticky sessions for data read paging).

> we can see that new logs are created on opt/hadoop/dfs/nn/current on the actine namenode on node01 but no new files . on the standby namenode no node02 - is it OK ??

Yes, this is normal. The new edit logs are redundantly written only when the NameNode is active. At all times the edits are primarily always written into the JournalNode directories.

View solution in original post

2 REPLIES 2

avatar
Mentor
> can we deploy the HttpFS role on more than one node? is it best practice?

Yes, the HttpFs service is an end-point for REST API access to HDFS, so you can deploy multiple instances and also consider load balancing (might need sticky sessions for data read paging).

> we can see that new logs are created on opt/hadoop/dfs/nn/current on the actine namenode on node01 but no new files . on the standby namenode no node02 - is it OK ??

Yes, this is normal. The new edit logs are redundantly written only when the NameNode is active. At all times the edits are primarily always written into the JournalNode directories.

avatar
Explorer
Thank you very much Harsh