httpfs is needed to support a centralized WebHDFS interface to an HA enable NN Cluster. This can be used by Hue or any other WebHDFS enabled client that needs to use a cluster configured with a High-Availability Namenode.
The installation is a piece of cake:
yum install hadoop-httpfs
But that's were the fun ends!!! Configuring is a whole other thing. It's not hard, if you know the right buttons to push. Unfortunately, the buttons and directions for doing this can be quite aloof.
The httpfs service is a tomcat application that relies on having the Hadoop libraries and configuration available, so it can resolve your HDP installation.
When you do the installation (above), a few items are installed.
From this point on, many of our changes are designed to "fix" the "hardcoded" implementations in the deployed scripts.
Adjust the /usr/hdp/current/hadoop-httpfs/sbin/httpfs.sh script
# Autodetect JAVA_HOME if not defined
if [ -e /usr/libexec/bigtop-detect-javahome ]; then
elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then
### Added to assist with locating the right configuration directory
### Remove the original HARD CODED Version reference... I mean, really???
exec /usr/hdp/current/hadoop-httpfs/sbin/httpfs.sh.distro "$@"
Now let's create a few symlinks to connect the pieces together
ln -s /etc/hadoop-httpfs/tomcat-deployment/conf conf
ln -s ../hadoop/libexec libexec
Like all the other Hadoop components, httpfs follows use *-env.sh files to control the startup environment. Above, in the httpfs.sh script we set the location of the configuration directory. That is used to find and load the httpfs-env.sh file we'll modified below.
# Add these to control and set the Catalina directories for starting and finding the httpfs application
# Set a log directory that matches your standards
# Set a tmp directory for httpfs to store interim files
That's it!! Now run it!
# To Stop