Malformed audit log for HDFS using ranger audit

adi1234 — Thu, 03 Jul 2025 23:02:09 GMT

Hi, We are trying to setup auditing for HDFS using ranger audit with configuration as in Cloudera Manager HDFS -> Configuration Tab in NameNode Logging Advanced Configuration Snippet as :

log4j.appender.RANGER_AUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.RANGER_AUDIT.File=/var/log/hadoop-hdfs/ranger-hdfs-audit.log log4j.appender.RANGER_AUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.RANGER_AUDIT.layout.ConversionPattern=%m%n log4j.logger.ranger.audit=INFO,RANGER_AUDIT,SYSAUDIT log4j.appender.SYSAUDIT=org.apache.log4j.net.SyslogAppender log4j.appender.SYSAUDIT.threshold=INFO

log4j.appender.SYSAUDIT.syslogHost=remote-host:port log4j.appender.SYSAUDIT.layout=org.apache.log4j.EnhancedPatternLayout log4j.appender.SYSAUDIT.layout.conversionPattern={ "log4j_tz_offset": "%d{Z}" }PR3N0RM %d{MMM dd HH:mm:ss} ${hostName}HDFS:%m%n log4j.appender.SYSAUDIT.filter.a=org.apache.log4j.varia.LevelRangeFilter log4j.appender.SYSAUDIT.filter.a.LevelMin=INFO log4j.appender.SYSAUDIT.filter.a.LevelMax=INFO

The log looks like

{"repoType":1,"repo":"cm_hdfs","reqUser":"hbase","evtTime":"2025-02-26 21:10:40.951","access":"delete","resource":"/hbase/archive/data/default/hbase_table/f11785d627e95e5f5a4bbd11065e1613/recovered.edits/17.seqid","resType":"path","action":"write","result":1,"agent":"hdfs","policy":-1,"reason":"/hbase/archive/data/default/hbase_table/f11785d627e95e5f5a4bbd11065e1613/recovered.edits","enforcer":"hadoop-acl","cliIP":"172.xx.xx.xx","reqData":"delete/CLI","agentHost":"ip-172-xx-xx-xx.ec2.internal","logType":"RangerAudit","id":"c47b16c5-25ae-44fe-8ae3-6ce05c4dc017-0","seq_num":1,"event_count":1,"event_dur_ms":1,"tags":[]}

However some of the log looks like

...ent/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_8bb79776e437aa37526bafe6f84f3a94","enforcer":"hadoop-acl","cliIP":"172.xx.xx.xx","agentHost":"ip-172-xx-xx-xx.ec2.internal","logType":"RangerAudit","id":"aee31510-b7cc-4400-82ac-5622ac39a1ad-0","seq_num":1,"event_count":1,"event_dur_ms":1,"tags":[]}

Any thoughts on why is this happening and what could cause this?

TIA.

Re: Malformed audit log for HDFS using ranger audit

VidyaSargur — Fri, 04 Jul 2025 08:30:32 GMT

Welcome to our community! To help you get the best possible answer, I have tagged our Ranger/HDFC experts, @vamsi_redd and @Scharan , who may be able to assist you further.

Please feel free to provide any additional information or details about your query. We hope that you will find a satisfactory solution to your question.

Re: Malformed audit log for HDFS using ranger audit

upadhyayk04 — Mon, 07 Jul 2025 10:20:39 GMT

Hello @adi1234

Thank you for contacting Cloudera community

Could you please confirm if you have followed the below document for setting up the configurations?

https://www.cloudera.com/blog/technical/auditing-to-external-systems-in-cdp-private-cloud-base.html

Also, which CDP version are you using?

question Re: Malformed audit log for HDFS using ranger audit in Support Questions

Malformed audit log for HDFS using ranger audit

Re: Malformed audit log for HDFS using ranger audit

Re: Malformed audit log for HDFS using ranger audit