question Re: custom cisco syslog to cef format in Support Questions

custom cisco syslog to cef format

cadrian90 — Wed, 24 Jul 2024 18:27:30 GMT

Hello,

We have custom syslog Cisco messages in the following format

<117>2024-07-23T14:09:56Z XXXXXXXXX : %FTD-5-430003: EventPriority: Low, DeviceUUID: xxxxxxxxxxxxx, InstanceID: 2, FirstPacketSecond: 2024-07-23T14:09:56Z, ConnectionID: 32322, AccessControlRuleAction: Allow, SrcIP: A.B.C.D, DstIP: A.B.C.D, SrcPort: 42308, DstPort: 24224, Protocol: tcp, IngressInterface: XXX, EgressInterface: XXX, IngressZone: XXX, EgressZone: YYY, IngressVRF: Global, EgressVRF: Global, ACPolicy: AAA-BBB,

We want to use NIFI to format to CEF ( common event format ). Any help which processors to use, please?

Re: custom cisco syslog to cef format

DianaTorres — Wed, 24 Jul 2024 18:50:50 GMT

@cadrian90 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @SAMSAL @MattWho who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: custom cisco syslog to cef format

SAMSAL — Thu, 25 Jul 2024 17:26:03 GMT

Hi @cadrian90 ,

Im not aware of direct way to do that in Nifi. I know there are services\processor like CEFFeader and ParseCEF used to consume CEF format but not to write as CEF. the good news is that you can write your custom code to create service or new processor to do that using Either Python or Java if you happen to know a way of doing using code.

Re: custom cisco syslog to cef format

MattWho — Fri, 26 Jul 2024 13:17:50 GMT

@cadrian90

I agree with @SAMSAL response. Typically the ConvertRecord processor is what would be used here. The processor support numerous record readers and numerous record writers. The GrokReader is what would be commonly used to parse unstructured data like your Cisco syslog messages. While the GrokReader has bulit in pattern file, you may fond yourself needing to define a custom pattern file for your specific data. You might find this other community post helpful here:
https://community.cloudera.com/t5/Support-Questions/ExtractGrok-processor-Writing-Regex-to-parse-Cisco-syslog/td-p/233095

Beyond above, this is where it becomes challenging since Apache NiFi only has a CEFReader and no CEFRecordSetWriter (perhaps you can raise an Apache Jira asking for this new reader and someone in the Apache community may be able to help)

There does exist a ScriptedRecordSetWriter that if you know how to scripted out the CEF format, maybe you can use that. I really would not be able to help there myself.
Maybe you can look into the CSVRecordSetWriter to see if selecting a custom format would facilitate an output like CEF. Again not something I have tried myself.

Hope this helps you with your use case journey.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: custom cisco syslog to cef format

DianaTorres — Tue, 30 Jul 2024 02:01:13 GMT

@cadrian90 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.