Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to deploy Flume Custom Interceptor

How to deploy Flume Custom Interceptor

New Contributor

We are needing to develop a custom interceptor to count the number of lines on the text file we are ingesting using Flume Spool Dir source.

 

Apache document discusses the ability to develop custom interceptors using its Interceptor interface.

 

After developing and compiling the custom interceptor code, documentation talkd about needing to copy the compiled jar to /usr/lib/flume-ng/lib/ so that it can be then used in the flume agent's configuration.

 

Although, i am not sure how it needs to work for the cloudera CDH. Where do we need to copy the jar file to. Copying to - /opt/cloudera/parcels/CDH-5.11.xxxx/lib/flume-ng/lib does not seem to work.

 

Please advise.

 

Thanks

Rahul

1 REPLY 1
Highlighted

Re: How to deploy Flume Custom Interceptor

Super Collaborator
The Flume service in CM has two "Plugin directories" configured by default: /var/lib/flume-ng/plugins.d and /usr/lib/flume-ng/plugins.d. You can follow the directions here to put your plugins in those directories:
http://flume.apache.org/FlumeUserGuide.html#directory-layout-for-plugins

-pd