Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

disable Flume syntax checker

avatar
New Contributor

Hallo,

 

I am trying to achieve in CM the following setup

 

Flume tier 1 - no sink - writes everything in a Kafka Channel

Flume tier 2 - no source - reads everything from the Kafka Channel above

 

with standalone (outside CM) agents it works, but when trying to add the configuration to the CM Agent it says "sink" missing. letting the definition of the sink empty - "a1.sinks = " - also doesn't work

 

Looks like somehow there is a syntax checker that doesn't allow me to input an "incomplete" Flume configuration.

 

The question is - is there a security valve or another configuration which controls this check ? I would turn it of, so I can run my configuration

 

Thanks

1 ACCEPTED SOLUTION

avatar
Contributor

Quaie,

I think you are actually ok on tier 2 (Kafka Channel to Sink - I do this all the time) even though CM does use the "warning" CSS which is not ideal if trying to edit the file directly. These warnings will be dismissable in CM 5.5

So for tier 1, currently this isn't possible but, you can pretty easily accomplish this by hacking around the agent file.

When flume starts, if a particular component doesn't have the confiugration correct, the agent will still run, it will just omit the misconfigured component.

So, perhaps you could just add the sink component but make the configuration wrong. 

 

Give that a shot and let me know how it goes.

 

Thanks

 

Jeff

View solution in original post

2 REPLIES 2

avatar
Contributor

Quaie,

I think you are actually ok on tier 2 (Kafka Channel to Sink - I do this all the time) even though CM does use the "warning" CSS which is not ideal if trying to edit the file directly. These warnings will be dismissable in CM 5.5

So for tier 1, currently this isn't possible but, you can pretty easily accomplish this by hacking around the agent file.

When flume starts, if a particular component doesn't have the confiugration correct, the agent will still run, it will just omit the misconfigured component.

So, perhaps you could just add the sink component but make the configuration wrong. 

 

Give that a shot and let me know how it goes.

 

Thanks

 

Jeff

avatar
New Contributor

Hallo Jeff,

 

I've configured a non-existing sink and, excepting the fact that the logs are filling with 

 

2015-11-06 08:18:53,195 ERROR org.apache.flume.SinkRunner: Unable to deliver event. Exception follows.

 

I found no other drawback.

 

Thanks for the solution