Archives of Support Questions (Read Only)

sharma24h · ‎10-05-2017

I have an Text file contain the text like

[16 Aug 2017 12:13:50,665] :INFO :UDPListener : UDP Listener ::: Receiver Node [ 0.0.0.0/3333 ] , Sender Node [ 20f:feb:1:0:0:0:0:10e ] , Message [ <30>Aug 16 12:13:50 as-pp-aa[1761]: %DAEMON-6-SNMP_TRAP_LINK_UP: ifIndex 669, ifAdminStatus up(1)] .

I want to split this line to Date : [16 Aug 2017 12:13:50,665] , Sender: [ 20f:feb:1:0:0:0:0:10e ] , Receve : [ 0.0.0.0/3333 ], and Message: [<30>Aug 16 12:13:50 as-pp-aa[1761]: %DAEMON-6-SNMP_TRAP_LINK_UP: ifIndex 669, ifAdminStatus up(1)] . Further i want to split the message part into some sub filed. Help me out is it possible to do with regular expression or i have to create the custom processor for this. please help me how i can do it ? i also want to save different filed to different text file. for further use of data analysis.

Shu_ashu · ‎10-05-2017

Hi @Sumit Sharma,

you can use replace text processor to extract and replace text as per your requirement.

Change the search value property to:-

(.+?)\s+:INFO.*Receiver Node\s+(\[.*\])\s+(?=,).*Sender Node\s+(\[.*\])\s+(?=,).*Message\s+(\[.*\])$

Change Replacement Value property to:-

Date: $1 ,sender: $3,Receve: $2, Message: $4

ReplaceText processor Configs:-

Input :-

[16 Aug 2017 12:13:50,665] :INFO :UDPListener : UDP Listener ::: Receiver Node [ 0.0.0.0/3333 ] , Sender Node [ 20f:feb:1:0:0:0:0:10e ] , Message [ <30>Aug 16 12:13:50 as-pp-aa[1761]: %DAEMON-6-SNMP_TRAP_LINK_UP: ifIndex 669, ifAdminStatus up(1)]

Output:-

Date: [16 Aug 2017 12:13:50,665] ,sender: [ 20f:feb:1:0:0:0:0:10e ],Receve: [ 0.0.0.0/3333 ], Message: [ <30>Aug 16 12:13:50 as-pp-aa[1761]: %DAEMON-6-SNMP_TRAP_LINK_UP: ifIndex 669, ifAdminStatus up(1)]

So this processor works dynamically according to the ff and replaces the content with your specifications.

View solution in original post

Shu_ashu · ‎10-11-2017

@Sumit Sharma,

Use UpdateAttribute processor before PutFile processor with below configurations.

Add new property to the processor by clicking + sign

filename as ${UUID()}

So this will replace the filename of the flowfile to UUID which is unique value all the time, it won't replace the file in your directory.

Cloudera Community

Archives of Support Questions (Read Only)

Read text file using GetFile Processor and SplitText processor to split each line filter out the text.