Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HA support by using HttpFs with Hue doesn't work

Solved Go to solution

HA support by using HttpFs with Hue doesn't work

Explorer

Hello,

 

I am using hue and I want to connect to a secure HDFS cluster. I have 3 Namenodes configured and I am running one httpfs server on each Namenode.I configured hue.ini to use all of them. I can access hdfs cluster from hue ui. However if I kill one of the httpfs server on one of the Namenode where hue initially connected to , hue is not able to switch to another httpfs server on another namenode.

 

Here is my hue.ini config :

[[hdfs_clusters]]
[[[default]]]
fs_defaultfs=hdfs://crd-dev-sor1p

logical_name=nn1

webhdfs_url=http://blitzhbase02-mnds3-1-crd.eng.sfdc.net:14000/webhdfs/v1

security_enabled=true

hadoop_conf_dir=/home/sfdc/current/bigdata-hadoop/hadoop/hadoop/etc/hadoop

 

[[[ha]]]
fs_defaultfs=hdfs://crd-dev-sor1p
webhdfs_url=http://blitzhbase02-mnds4-1-crd.eng.sfdc.net:14000/webhdfs/v1
security_enabled=true

 

[[[ha1]]]
fs_defaultfs=hdfs://crd-dev-sor1p
webhdfs_url=http://blitzhbase02-dnds2-2-crd.eng.sfdc.net:14000/webhdfs/v1
security_enabled=true

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HA support by using HttpFs with Hue doesn't work

This is expected, the current design for HttpFs HA comes with using a
balancer in front of the HttpFs servers.

Only for the YARN Resource Manager Hue is operating a failover itself.
3 REPLIES 3

Re: HA support by using HttpFs with Hue doesn't work

This is expected, the current design for HttpFs HA comes with using a
balancer in front of the HttpFs servers.

Only for the YARN Resource Manager Hue is operating a failover itself.

Re: HA support by using HttpFs with Hue doesn't work

Explorer

Hi Romain,

 

Thanks for the reply.

 

Yes I could setup hue to do failover itself for yarn RM.

 

HttpFS for hue is already acting as a balancer for Namenodes , now setting up balancer again for HttpFS is like setting up balancer for balancer.

 

Any guide on setting up balancer before Httpfs for hue and corresponding sample hue.ini for that setup ?

 

 

 

 

Highlighted

Re: HA support by using HttpFs with Hue doesn't work

Contributor

Maybe it could be better to launch httpfs on the node with the Hue? In such deployment hue+httpfs node will communicate with the HDFS datanodes directly and not through other server (the Hue machine itself will be the HTTP proxy).

 

But I guess balancing of the multiple httpfs proxies around the cluster would be usefull even in this case - for high available WebHDFS interface for end users.