question disable Flume syntax checker in Archives of Support Questions (Read Only)

disable Flume syntax checker

quaie — Fri, 16 Sep 2022 09:43:57 GMT

Hallo,

I am trying to achieve in CM the following setup

Flume tier 1 - no sink - writes everything in a Kafka Channel

Flume tier 2 - no source - reads everything from the Kafka Channel above

with standalone (outside CM) agents it works, but when trying to add the configuration to the CM Agent it says "sink" missing. letting the definition of the sink empty - "a1.sinks = " - also doesn't work

Looks like somehow there is a syntax checker that doesn't allow me to input an "incomplete" Flume configuration.

The question is - is there a security valve or another configuration which controls this check ? I would turn it of, so I can run my configuration

Thanks

Re: disable Flume syntax checker

jholoman — Wed, 21 Oct 2015 16:58:59 GMT

Quaie,

I think you are actually ok on tier 2 (Kafka Channel to Sink - I do this all the time) even though CM does use the "warning" CSS which is not ideal if trying to edit the file directly. These warnings will be dismissable in CM 5.5

So for tier 1, currently this isn't possible but, you can pretty easily accomplish this by hacking around the agent file.

When flume starts, if a particular component doesn't have the confiugration correct, the agent will still run, it will just omit the misconfigured component.

So, perhaps you could just add the sink component but make the configuration wrong.

Give that a shot and let me know how it goes.

Thanks

Jeff

Re: disable Flume syntax checker

quaie — Fri, 06 Nov 2015 07:21:24 GMT

Hallo Jeff,

I've configured a non-existing sink and, excepting the fact that the logs are filling with

2015-11-06 08:18:53,195 ERROR org.apache.flume.SinkRunner: Unable to deliver event. Exception follows.

I found no other drawback.

Thanks for the solution