Reply
New Contributor
Posts: 3
Registered: ‎03-18-2016

How to deploy Flume Custom Interceptor

We are needing to develop a custom interceptor to count the number of lines on the text file we are ingesting using Flume Spool Dir source.

 

Apache document discusses the ability to develop custom interceptors using its Interceptor interface.

 

After developing and compiling the custom interceptor code, documentation talkd about needing to copy the compiled jar to /usr/lib/flume-ng/lib/ so that it can be then used in the flume agent's configuration.

 

Although, i am not sure how it needs to work for the cloudera CDH. Where do we need to copy the jar file to. Copying to - /opt/cloudera/parcels/CDH-5.11.xxxx/lib/flume-ng/lib does not seem to work.

 

Please advise.

 

Thanks

Rahul

Cloudera Employee
Posts: 234
Registered: ‎01-09-2014

Re: How to deploy Flume Custom Interceptor

The Flume service in CM has two "Plugin directories" configured by default: /var/lib/flume-ng/plugins.d and /usr/lib/flume-ng/plugins.d. You can follow the directions here to put your plugins in those directories:
http://flume.apache.org/FlumeUserGuide.html#directory-layout-for-plugins

-pd
Announcements