question Re: Ranger audit log for Stream Messaging Manager service (SMM) in Support Questions

Ranger audit log for Stream Messaging Manager service (SMM)

snm1523 — Tue, 21 Apr 2026 06:32:32 GMT

Hello All,

On CDP PB, like we send ranger audit logs of services thats support Ranger plugin to HDFS, can we also send logs of Stream Messaging Manager service to HDFS?

I tried to look for config to achieve this, however, I could find only 2 configs (listed below) that allow us to configure the local path. However, there was nothing found to send these logs to HDFS.

Ranger Streams Messaging Manager Plugin Audit Hdfs Spool Directory Path

Ranger Streams Messaging Manager Plugin Audit Solr Spool Directory Path

Any help / guidance will be highly appreciated.

Thanks
snm1523

Re: Ranger audit log for Stream Messaging Manager service (SMM)

Rajat_710 — Mon, 15 Apr 2024 16:11:24 GMT

Hi @snm1523 those configs are for spool directories, these configs won't help you.

The audit logs are stored in HDFS, It's Solr collections that will store them in HDFS, if your solr is configured to store them in HDFS then by default all the auditing will happen in HDFS. You can refer to the below doc to check where and how solr collections are stored.

https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/security-ranger-auditing/topics/security-ranger-audit-migrating-storage-data-location.html

Re: Ranger audit log for Stream Messaging Manager service (SMM)

snm1523 — Wed, 17 Apr 2024 08:30:54 GMT

Hello @Rajat_710.,

Thank you for the response.

Will definitely go through the article. However, have got a follow up question.

I also noticed on master servers (local location configured using relevant properties), that the audit logs (HDFS and SOLR) are not getting purged automatically once they are sent to HDFS. Instead we have to run a cron job that triggers a shell script which clears the logs the from the location. If the job is stopped even for 15 mins, we will see a pile of logs at that location.

Any suggestions if we have missed anything?

Thanks

snm1523

Re: Ranger audit log for Stream Messaging Manager service (SMM)

Rajat_710 — Wed, 17 Apr 2024 08:51:48 GMT

Hi @snm1523 , correct you have to manually delete those files in HDFS, we are aware of it and currently we are working on that issue.

Re: Ranger audit log for Stream Messaging Manager service (SMM)

snm1523 — Wed, 17 Apr 2024 11:12:21 GMT

Thank you for the confirmation, @Rajat_710.

Is there a Cloudera article / blog post / documentation informing this as a known issue or WIP that I can use as reference in our internal discussions. Will need something to propose and support manual deletion of HDFS and SOLR audit logs from local location.

Thanks
snm1523

Re: Ranger audit log for Stream Messaging Manager service (SMM)

Rajat_710 — Mon, 22 Apr 2024 06:25:36 GMT

Hi, @snm1523 you can find some info here,

https://community.cloudera.com/t5/Support-Questions/Ranger-audit-log-to-hdfs/m-p/323456/highlight/true

Re: Ranger audit log for Stream Messaging Manager service (SMM)

snm1523 — Mon, 22 Apr 2024 09:49:08 GMT

Thank you for the reference, @Rajat_710 .

That link is referencing to a community post where someone has discussed about logs in HDFS.

However, here I am more keen towards logs that are stored in local location and not getting deleted even after they are moved to HDFS. Since, you mentioned that this has to be manually deleted and Cloudera is currently working on implementing automatic purge of these logs from local, would it be possible to kindly share a reference article or a Cloudera blog post or a known issue reference where this has been mentioned / recorded.

Thanks
snm1523

Re: Ranger audit log for Stream Messaging Manager service (SMM)

Rajat_710 — Mon, 22 Apr 2024 10:13:47 GMT

Hi @snm1523 right now i dont any doc for this.

Re: Ranger audit log for Stream Messaging Manager service (SMM)

snm1523 — Wed, 01 May 2024 10:58:01 GMT

Hello @Rajat_710 ,

This is a last follow up query on this thread.

Any clue how we configure Streams Messaging Manager server to send audit logs to HDFS and / or Solr (just like other services) and then they would get archived to /archive directory from where we will manually delete them?

I am referring to logs that get stored under below location locally (not HDFS)
/var/log/<service name>/audit/<HDFS or Solr>/spool/

Thanks
snm1523

Re: Ranger audit log for Stream Messaging Manager service (SMM)

Rajat_710 — Wed, 15 May 2024 08:27:27 GMT

@snm1523 apologies for the late response, logs stored under "/var/log/<service name>/audit/<HDFS or Solr>/spool/" location means something is wrong with the plugin service.

By default, audit logs will be stored in HDFS but when the end service is not responding properly then only the spool files will be created under here. It will store on local only when SMM is not responding

Re: Ranger audit log for Stream Messaging Manager service (SMM)

snm1523 — Thu, 16 May 2024 09:35:43 GMT

Thank you for the explanation @Rajat_710. This helps.

So once the logs from spool are processed they are moved to archive and hence, safe to be deleted, correct?

Secondly, for SMM, logs are not going to HDFS at all. Is there a way to configure audit logs of SMM to go to HDFS like we are sending for other services? I did not see the configuration anywhere in SMM to enable this.

Thanks
snm1523

Re: Ranger audit log for Stream Messaging Manager service (SMM)

Rajat_710 — Fri, 26 Jul 2024 12:49:48 GMT

no, you cannot configure audit logs of SMM to go to HDFS