Support Questions

Find answers, ask questions, and share your expertise

Malformed audit log for HDFS using ranger audit

avatar
Visitor

Hi, We are trying to setup auditing for HDFS using ranger audit with configuration as in Cloudera Manager HDFS -> Configuration Tab in NameNode Logging Advanced Configuration Snippet as :

log4j.appender.RANGER_AUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.RANGER_AUDIT.File=/var/log/hadoop-hdfs/ranger-hdfs-audit.log log4j.appender.RANGER_AUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.RANGER_AUDIT.layout.ConversionPattern=%m%n log4j.logger.ranger.audit=INFO,RANGER_AUDIT,SYSAUDIT log4j.appender.SYSAUDIT=org.apache.log4j.net.SyslogAppender log4j.appender.SYSAUDIT.threshold=INFO

log4j.appender.SYSAUDIT.syslogHost=remote-host:port log4j.appender.SYSAUDIT.layout=org.apache.log4j.EnhancedPatternLayout log4j.appender.SYSAUDIT.layout.conversionPattern={ "log4j_tz_offset": "%d{Z}" }PR3N0RM %d{MMM dd HH:mm:ss} ${hostName}HDFS:%m%n log4j.appender.SYSAUDIT.filter.a=org.apache.log4j.varia.LevelRangeFilter log4j.appender.SYSAUDIT.filter.a.LevelMin=INFO log4j.appender.SYSAUDIT.filter.a.LevelMax=INFO

The log looks like

{"repoType":1,"repo":"cm_hdfs","reqUser":"hbase","evtTime":"2025-02-26 21:10:40.951","access":"delete","resource":"/hbase/archive/data/default/hbase_table/f11785d627e95e5f5a4bbd11065e1613/recovered.edits/17.seqid","resType":"path","action":"write","result":1,"agent":"hdfs","policy":-1,"reason":"/hbase/archive/data/default/hbase_table/f11785d627e95e5f5a4bbd11065e1613/recovered.edits","enforcer":"hadoop-acl","cliIP":"172.xx.xx.xx","reqData":"delete/CLI","agentHost":"ip-172-xx-xx-xx.ec2.internal","logType":"RangerAudit","id":"c47b16c5-25ae-44fe-8ae3-6ce05c4dc017-0","seq_num":1,"event_count":1,"event_dur_ms":1,"tags":[]}

 

However some of the log looks like

...ent/user/hue/.cloudera_manager_hive_metastore_canary/cloudera_manager_metastore_canary_test_catalog_hive_HIVEMETASTORE_8bb79776e437aa37526bafe6f84f3a94","enforcer":"hadoop-acl","cliIP":"172.xx.xx.xx","agentHost":"ip-172-xx-xx-xx.ec2.internal","logType":"RangerAudit","id":"aee31510-b7cc-4400-82ac-5622ac39a1ad-0","seq_num":1,"event_count":1,"event_dur_ms":1,"tags":[]}

Any thoughts on why is this happening and what could cause this?

TIA.

 

2 REPLIES 2

avatar
Community Manager

Welcome to our community! To help you get the best possible answer, I have tagged our Ranger/HDFC experts, @vamsi_redd and @Scharan , who may be able to assist you further.

Please feel free to provide any additional information or details about your query. We hope that you will find a satisfactory solution to your question.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Collaborator

Hello @adi1234 

Thank you for contacting Cloudera community

Could you please confirm if you have followed the below document for setting up the configurations?

https://www.cloudera.com/blog/technical/auditing-to-external-systems-in-cdp-private-cloud-base.html

Also, which CDP version are you using?