Created on 07-01-2015 12:03 AM - edited 09-16-2022 02:32 AM
I'm searching for the recommanded configuration for Flume - HDFS Sink when we are using HDFS in HA.
In fact, each time that we restart the cluster/ the nodename fails the active nodename changes and flume fails since is
asking informations on the standby node.
Created 10-16-2017 01:54 AM
Created 10-16-2017 02:20 AM
Created 02-23-2021 04:41 AM
Yes, you can download the hdfs client configuration from Cloudera Manager, but this is not possible always, when you are working on different department or any bureaucratic issue... And if you make any change HDFS configuration, you must download this configuration again. Is not a scalable solution in a big environments, the best solution is working on the same cluster (gateway if is possible), but if you have an external Flume Agents, there no exist a properly and scalable solution I think.