Reply
sbn
Explorer
Posts: 8
Registered: ‎02-16-2017
Accepted Solution

webhdfs way to always post to ACTIVE namenode

i have a logging setup where i have a fluent/webhdfs host, for writing logs  from fluent to hdfs.

 

a problem is i can only write to the ACTIVE namenode, this sets some requirements to the logic in the fluent/webhdfs host  that i would like to avoid.

 

is there a way to make sure youre always posting to the ACTIVE namenode?

 

Highlighted
New Contributor
Posts: 3
Registered: ‎05-17-2017

Re: webhdfs way to always post to ACTIVE namenode

It is a fluentd issue, have you checked fluentd documentation ?

Take a look to https://github.com/fluent/fluent-plugin-webhdfs :

 

<match access.**>
  @type            webhdfs
  namenode         master1.your.cluster.local:50070
  standby_namenode master2.your.cluster.local:50070
  path             /path/on/hdfs/access.log.%Y%m%d_%H.log
</match>

 

 

Announcements