Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

webhdfs way to always post to ACTIVE namenode

Solved Go to solution

webhdfs way to always post to ACTIVE namenode

Explorer

i have a logging setup where i have a fluent/webhdfs host, for writing logs  from fluent to hdfs.

 

a problem is i can only write to the ACTIVE namenode, this sets some requirements to the logic in the fluent/webhdfs host  that i would like to avoid.

 

is there a way to make sure youre always posting to the ACTIVE namenode?

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: webhdfs way to always post to ACTIVE namenode

New Contributor

It is a fluentd issue, have you checked fluentd documentation ?

Take a look to https://github.com/fluent/fluent-plugin-webhdfs :

 

<match access.**>
  @type            webhdfs
  namenode         master1.your.cluster.local:50070
  standby_namenode master2.your.cluster.local:50070
  path             /path/on/hdfs/access.log.%Y%m%d_%H.log
</match>

 

 

1 REPLY 1

Re: webhdfs way to always post to ACTIVE namenode

New Contributor

It is a fluentd issue, have you checked fluentd documentation ?

Take a look to https://github.com/fluent/fluent-plugin-webhdfs :

 

<match access.**>
  @type            webhdfs
  namenode         master1.your.cluster.local:50070
  standby_namenode master2.your.cluster.local:50070
  path             /path/on/hdfs/access.log.%Y%m%d_%H.log
</match>