Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS Federation - Namenode leader election for more than single web apps on top of it

HDFS Federation - Namenode leader election for more than single web apps on top of it

New Contributor

I want to run our web apps (like more than 100 count) on top of hdfs federation ? With so many namenodes as active in federation how all requests coming through these web apps will be handled ?

I mean, first of all is it possible to do so ?

If yes than how it will be decided that which namenode is going to serve which requests for files read/write from all of this 100+ web apps.I want some architectural level help to achieve this.

1 REPLY 1
Highlighted

Re: HDFS Federation - Namenode leader election for more than single web apps on top of it

Namenode Federation is the assignment of directories to a pair of HA NN's, like a mount point on linux. The process of directing clients to the right Namenode is the responsibility of the HDFS client. And if you're using the standard HDFS libraries, then having the "properly" configured hdfs-site.xml file in the path of the java client will handle that transparently.

This config file (hdfs-site.xml) will contain all the NN HA and Federation settings needed.

Don't have an account?
Coming from Hortonworks? Activate your account here