About snm1523

snm1523 · ‎05-20-2024

Hello, We have recently migrated from HDP to CDP 7.1.9 and new Knox topologies created as per Cloudera recommendations (cdp-proxy, cdp-proxy-api and cdp-proxy-token) are in place and configured. However, for some reason the topologies from HDP environment are also inherited and are visible on Knox UI, though are not active / used. Kindly advise on below points: 1. Is this a normal behaviour that Knox topologies get inherited OR we missed something? 2. How do we delete these unwanted topologies so they don't appear in Knox UI. 3. There are some topologies that we would still need, however, few services like Ambari and Zeppelin are not needed. How do I remove these from an existing topology? Thanks snm1523

snm1523 · ‎05-16-2024

Thank you for the explanation @Rajat_710. This helps. So once the logs from spool are processed they are moved to archive and hence, safe to be deleted, correct? Secondly, for SMM, logs are not going to HDFS at all. Is there a way to configure audit logs of SMM to go to HDFS like we are sending for other services? I did not see the configuration anywhere in SMM to enable this. Thanks snm1523

snm1523 · ‎05-15-2024

Hello, Any clue on how we configure Streams Messaging Manager server to send audit logs to HDFS and / or Solr (just like other services) and then they would get archived to /archive directory from where we can manually delete them? I am referring to logs that get stored under below location locally (not HDFS) /var/log/streams-messaging-manager/audit/<hdfs or solr>/spool Thanks snm1523

snm1523 · ‎05-13-2024

Thank you for the response @RAGHUY. Would these be the only steps we need to perform or based on your experience, you have identified anything additional which is not mentioned in the documentation you shared. Thanks snm1523

snm1523 · ‎05-10-2024

Hi All, I am aware once we upgrade CDP PB from x to y, we need to finalise HDFS upgrade if it is required. However, recently in the team got to know from someone who has done the upgrade that we also need to finalize Kafka upgrade along with HDFS. What needs to be done with finalising Kafka upgrade? Is it a newer thing in latest CDP version or something I have always missed on previous Kafka clusters? Haven't had a need of doing this. Thanks snm1523

snm1523 · ‎05-01-2024

Hello @Rajat_710 , This is a last follow up query on this thread. Any clue how we configure Streams Messaging Manager server to send audit logs to HDFS and / or Solr (just like other services) and then they would get archived to /archive directory from where we will manually delete them? I am referring to logs that get stored under below location locally (not HDFS) /var/log/<service name>/audit/<HDFS or Solr>/spool/ Thanks snm1523

snm1523 · ‎04-22-2024

Thank you for the reference, @Rajat_710 . That link is referencing to a community post where someone has discussed about logs in HDFS. However, here I am more keen towards logs that are stored in local location and not getting deleted even after they are moved to HDFS. Since, you mentioned that this has to be manually deleted and Cloudera is currently working on implementing automatic purge of these logs from local, would it be possible to kindly share a reference article or a Cloudera blog post or a known issue reference where this has been mentioned / recorded. Thanks snm1523

snm1523 · ‎04-17-2024

Thank you for the confirmation, @Rajat_710. Is there a Cloudera article / blog post / documentation informing this as a known issue or WIP that I can use as reference in our internal discussions. Will need something to propose and support manual deletion of HDFS and SOLR audit logs from local location. Thanks snm1523

snm1523 · ‎04-17-2024

Hello @Rajat_710., Thank you for the response. Will definitely go through the article. However, have got a follow up question. I also noticed on master servers (local location configured using relevant properties), that the audit logs (HDFS and SOLR) are not getting purged automatically once they are sent to HDFS. Instead we have to run a cron job that triggers a shell script which clears the logs the from the location. If the job is stopped even for 15 mins, we will see a pile of logs at that location. Any suggestions if we have missed anything? Thanks snm1523

snm1523 · ‎04-11-2024

Hello All, On CDP PB, like we send ranger audit logs of services thats support Ranger plugin to HDFS, can we also send logs of Stream Messaging Manager service to HDFS? I tried to look for config to achieve this, however, I could find only 2 configs (listed below) that allow us to configure the local path. However, there was nothing found to send these logs to HDFS. Ranger Streams Messaging Manager Plugin Audit Hdfs Spool Directory Path Ranger Streams Messaging Manager Plugin Audit Solr Spool Directory Path Any help / guidance will be highly appreciated. Thanks snm1523

Online	Offline
Last Visited	‎11-07-2025 08:17 AM

Member Since	‎10-29-2015 07:36 PM
Last Visited	‎11-07-2025 08:17 AM
Posts	128
Kudos received	31

Cloudera Community

Re: YARN and HDFS monitoring via Grafana

Re: Enable Admin account for Cloudera Manager

Re: Datanode not starting: SIGTERM error

Re: MKDirs failed to create file

Delete unwanted knox topologies - CDP Private Clou...

Re: Ranger audit log for Stream Messaging Manager ...

Streams Messaging Server audit logs - CDP Private ...

Re: Finalise Kafka upgrade - CDP Private Cloud

Finalise Kafka upgrade - CDP Private Cloud

Re: Ranger audit log for Stream Messaging Manager ...

Re: Ranger audit log for Stream Messaging Manager ...

Re: Ranger audit log for Stream Messaging Manager ...

Re: Ranger audit log for Stream Messaging Manager ...

Ranger audit log for Stream Messaging Manager serv...