Install, config and start HttpFS in HDP 3.1.0

[#1]

Ambari; 2.7.3.0

HDP: 3.1.0

Ubuntu 16.04

Hi there,

I want to use HttpFS. But when I run command:

hdfs httpfs start

But I faced this error:

Exception in thread "main" java.io.IOException: Unable to initialize WebAppContext

at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1156)

at org.apache.hadoop.fs.http.server.HttpFSServerWebServer.start(HttpFSServerWebServer.java:132)

at org.apache.hadoop.fs.http.server.HttpFSServerWebServer.main(HttpFSServerWebServer.java:165)

Caused by: java.lang.RuntimeException: Could not read HttpFS signature secret file: /usr/hdp/3.1.0.0-78/hadoop/conf/httpfs-signature.secret

at org.apache.hadoop.fs.http.server.HttpFSAuthenticationFilter.getConfiguration(HttpFSAuthenticationFilter.java:95)

Can you help me? Thank you.

[#2]

Hi @LUAN HA,

We had the same issue.

We have added to HDFS Ambari config "hadoop-env template" extra variable:

export HTTPFS_CONFIG=/etc/hadoop-httpfs/conf

In dir:

/etc/hadoop-httpfs/conf 

We have placed secret file:

httpfs-signature.secret

Reboot service from httpfs user:

hdfs --daemon stop httpfs
hdfs --daemon start httpfs

Otherwise httpfs is using default client hadoop config in example: /usr/hdp/3.1.0.0-78/hadoop/conf as the config dir which will be overwritten if you reboot services using Ambari.