Malformed audit log for HDFS using ranger audit
Hi, We are trying to setup auditing for HDFS using ranger audit with configuration as in Cloudera Manager HDFS -> Configuration Tab in NameNode Logging Advanced Configuration Snippet as :
log4j.appender.RANGER_AUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.RANGER_AUDIT.File=/var/log/hadoop-hdfs/ranger-hdfs-audit.log log4j.appender.RANGER_AUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.RANGER_AUDIT.layout.ConversionPattern=%m%n log4j.logger.ranger.audit=INFO,RANGER_AUDIT,SYSAUDIT log4j.appender.SYSAUDIT=org.apache.log4j.net.SyslogAppender log4j.appender.SYSAUDIT.threshold=INFO
log4j.appender.SYSAUDIT.syslogHost=remote-host:port log4j.appender.SYSAUDIT.layout=org.apache.log4j.EnhancedPatternLayout log4j.appender.SYSAUDIT.layout.conversionPattern={ "log4j_tz_offset": "%d{Z}" }PR3N0RM %d{MMM dd HH:mm:ss} ${hostName}HDFS:%m%n log4j.appender.SYSAUDIT.filter.a=org.apache.log4j.varia.LevelRangeFilter log4j.appender.SYSAUDIT.filter.a.LevelMin=INFO log4j.appender.SYSAUDIT.filter.a.LevelMax=INFO
The log looks like
{"repoType":1,"repo":"cm_hdfs","reqUser":"hbase","evtTime":"2025-02-26 21:10:40.951","access":"delete","resource":"/hbase/archive/data/default/hbase_table/f11785d627e95e5f5a4bbd11065e1613/recovered.edits/17.seqid","resType":"path","action":"write","result":1,"agent":"hdfs","policy":-1,"reason":"/hbase/archive/data/default/hbase_table/f11785d627e95e5f5a4bbd11065e1613/recovered.edits","enforcer":"hadoop-acl","cliIP":"172.xx.xx.xx","reqData":"delete/CLI","agentHost":"ip-172-xx-xx-xx.ec2.internal","logType":"RangerAudit","id":"c47b16c5-25ae-44fe-8ae3-6ce05c4dc017-0","seq_num":1,"event_count":1,"event_dur_ms":1,"tags":[]}
However some of the log looks like
...ent/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_8bb79776e437aa37526bafe6f84f3a94","enforcer":"hadoop-acl","cliIP":"172.xx.xx.xx","agentHost":"ip-172-xx-xx-xx.ec2.internal","logType":"RangerAudit","id":"aee31510-b7cc-4400-82ac-5622ac39a1ad-0","seq_num":1,"event_count":1,"event_dur_ms":1,"tags":[]}
Any thoughts on why is this happening and what could cause this?
TIA.
