10-13-2015 11:08 PM
I am trying to achieve in CM the following setup
Flume tier 1 - no sink - writes everything in a Kafka Channel
Flume tier 2 - no source - reads everything from the Kafka Channel above
with standalone (outside CM) agents it works, but when trying to add the configuration to the CM Agent it says "sink" missing. letting the definition of the sink empty - "a1.sinks = " - also doesn't work
Looks like somehow there is a syntax checker that doesn't allow me to input an "incomplete" Flume configuration.
The question is - is there a security valve or another configuration which controls this check ? I would turn it of, so I can run my configuration
10-21-2015 09:58 AM
I think you are actually ok on tier 2 (Kafka Channel to Sink - I do this all the time) even though CM does use the "warning" CSS which is not ideal if trying to edit the file directly. These warnings will be dismissable in CM 5.5
So for tier 1, currently this isn't possible but, you can pretty easily accomplish this by hacking around the agent file.
When flume starts, if a particular component doesn't have the confiugration correct, the agent will still run, it will just omit the misconfigured component.
So, perhaps you could just add the sink component but make the configuration wrong.
Give that a shot and let me know how it goes.
11-05-2015 11:21 PM
I've configured a non-existing sink and, excepting the fact that the logs are filling with
2015-11-06 08:18:53,195 ERROR org.apache.flume.SinkRunner: Unable to deliver event. Exception follows.
I found no other drawback.
Thanks for the solution