Support Questions
Find answers, ask questions, and share your expertise

flume is missing "search_replace" interceptor

Highlighted

flume is missing "search_replace" interceptor

Hi All,

Looks like search_replace interceptor is missing in HDP 2.6. Can anyone please advise an alternate method to search_replace interceptor in flume ?

3 REPLIES 3

Re: flume is missing "search_replace" interceptor

Contributor

Hi, Did you try this ?

agent.sources.localsource.interceptors = search-replace
agent.sources.localsource.interceptors.search-replace.type = search_replace
agent.sources.interceptors.search-replace.searchPattern = ^INFO:
agent.sources.interceptors.search-replace.replaceString = Log msg:

Highlighted

Re: flume is missing "search_replace" interceptor

Hi @dthakkar, Im using below piece of config which works flawlessly in apache flume 1.7 but HDP flume throws error saying class not found for search_replace. Even HDP Flume documentation doesn't mentions search_replace

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/ds_flume/FlumeUserGuide.html

mytest.sources.src.interceptors = s0 s1 s2 s3
mytest.sources.src.interceptors.s0.type = host
mytest.sources.src.interceptors.so.useIP = false
mytest.sources.src.interceptors.s1.type = search_replace
mytest.sources.src.interceptors.s1.searchPattern = ^\\{
mytest.sources.src.interceptors.s1.replaceString =  \\"
mytest.sources.src.interceptors.s2.type = search_replace
mytest.sources.src.interceptors.s2.searchPattern = \\}
mytest.sources.src.interceptors.s2.replaceString =
mytest.sources.src.interceptors.s3.type = search_replace
mytest.sources.src.interceptors.s3.searchPattern = \\,\\{


<br>
Highlighted

Re: flume is missing "search_replace" interceptor

Update !

We created our own jar file for search_replace from apache flume 1.6 code with minor changes.

Now its working as expected.