Support Questions
Find answers, ask questions, and share your expertise

webhdfs way to always post to ACTIVE namenode

Solved Go to solution
Highlighted

webhdfs way to always post to ACTIVE namenode

Explorer

i have a logging setup where i have a fluent/webhdfs host, for writing logs  from fluent to hdfs.

 

a problem is i can only write to the ACTIVE namenode, this sets some requirements to the logic in the fluent/webhdfs host  that i would like to avoid.

 

is there a way to make sure youre always posting to the ACTIVE namenode?

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: webhdfs way to always post to ACTIVE namenode

New Contributor

It is a fluentd issue, have you checked fluentd documentation ?

Take a look to https://github.com/fluent/fluent-plugin-webhdfs :

 

<match access.**>
  @type            webhdfs
  namenode         master1.your.cluster.local:50070
  standby_namenode master2.your.cluster.local:50070
  path             /path/on/hdfs/access.log.%Y%m%d_%H.log
</match>

 

 

View solution in original post

1 REPLY 1
Highlighted

Re: webhdfs way to always post to ACTIVE namenode

New Contributor

It is a fluentd issue, have you checked fluentd documentation ?

Take a look to https://github.com/fluent/fluent-plugin-webhdfs :

 

<match access.**>
  @type            webhdfs
  namenode         master1.your.cluster.local:50070
  standby_namenode master2.your.cluster.local:50070
  path             /path/on/hdfs/access.log.%Y%m%d_%H.log
</match>

 

 

View solution in original post