Reply
New Contributor
Posts: 2
Registered: ‎10-13-2015
Accepted Solution

disable Flume syntax checker

Hallo,

 

I am trying to achieve in CM the following setup

 

Flume tier 1 - no sink - writes everything in a Kafka Channel

Flume tier 2 - no source - reads everything from the Kafka Channel above

 

with standalone (outside CM) agents it works, but when trying to add the configuration to the CM Agent it says "sink" missing. letting the definition of the sink empty - "a1.sinks = " - also doesn't work

 

Looks like somehow there is a syntax checker that doesn't allow me to input an "incomplete" Flume configuration.

 

The question is - is there a security valve or another configuration which controls this check ? I would turn it of, so I can run my configuration

 

Thanks

Cloudera Employee
Posts: 21
Registered: ‎07-08-2013

Re: disable Flume syntax checker

Quaie,

I think you are actually ok on tier 2 (Kafka Channel to Sink - I do this all the time) even though CM does use the "warning" CSS which is not ideal if trying to edit the file directly. These warnings will be dismissable in CM 5.5

So for tier 1, currently this isn't possible but, you can pretty easily accomplish this by hacking around the agent file.

When flume starts, if a particular component doesn't have the confiugration correct, the agent will still run, it will just omit the misconfigured component.

So, perhaps you could just add the sink component but make the configuration wrong. 

 

Give that a shot and let me know how it goes.

 

Thanks

 

Jeff

Highlighted
New Contributor
Posts: 2
Registered: ‎10-13-2015

Re: disable Flume syntax checker

Hallo Jeff,

 

I've configured a non-existing sink and, excepting the fact that the logs are filling with 

 

2015-11-06 08:18:53,195 ERROR org.apache.flume.SinkRunner: Unable to deliver event. Exception follows.

 

I found no other drawback.

 

Thanks for the solution

Announcements
New solutions