webhdfs way to always post to ACTIVE namenode

[#1]

i have a logging setup where i have a fluent/webhdfs host, for writing logs  from fluent to hdfs.

a problem is i can only write to the ACTIVE namenode, this sets some requirements to the logic in the fluent/webhdfs host  that i would like to avoid.

is there a way to make sure youre always posting to the ACTIVE namenode?

[#2 · Accepted solution]

It is a fluentd issue, have you checked fluentd documentation ?

Take a look to https://github.com/fluent/fluent-plugin-webhdfs :

<match access.**>
  @type            webhdfs
  namenode         master1.your.cluster.local:50070
  standby_namenode master2.your.cluster.local:50070
  path             /path/on/hdfs/access.log.%Y%m%d_%H.log
</match>