Created 05-23-2024 04:09 AM
Hello,
Has anyone encountered an issue were Knox is not writing audit logs of specific topology. We have below topologies created including few of them migrated from HDP, however, necessary modifications were done and are listed in Knox UI.
Knox is successfully writing Ranger audit logs in HDFS for only cdp* topologies which were created by Cloudera during setup of Knox service and not for other. Written logs are visible in access tab of Audit section in Ranger Admin UI. We have total of 3 clusters and this is the case 2 clusters, for 1 cluster everything works fine. I have compared the configuration and also topology xmls and all seems correct (except for instance details which is obvious).
Would it be anything related to Ranger or Solr configuration for Knox? However, if that is case it should be applicable to all topologies of Knox, why only non-default ones?
Please help with suggestions / things to check / troubleshoot.
Thanks
snm1523
Created 05-27-2024 10:37 PM
@snm1523HealthCareGov wrote:Hello,
Has anyone encountered an issue were Knox is not writing audit logs of specific topology. We have below topologies created including few of them migrated from HDP, however, necessary modifications were done and are listed in Knox UI.
- cdp-proxy
- cdp-proxy-api
- cdp-proxy-token
- health
- tokenexchange
- user1 - created for user group
- topo1 - created for user group and migrated from HDP
- topo2 - created for user group and migrated from HDP
- app - Used by app accounts
Knox is successfully writing Ranger audit logs in HDFS for only cdp* topologies which were created by Cloudera during setup of Knox service and not for other. Written logs are visible in access tab of Audit section in Ranger Admin UI. We have total of 3 clusters and this is the case 2 clusters, for 1 cluster everything works fine. I have compared the configuration and also topology xmls and all seems correct (except for instance details which is obvious).
Would it be anything related to Ranger or Solr configuration for Knox? However, if that is case it should be applicable to all topologies of Knox, why only non-default ones?
Please help with suggestions / things to check / troubleshoot.
Thanks
snm1523
Hello,
The issue you're describing where Knox isn't writing audit logs for specific, non-default topologies is a known problem in CDP Private Cloud. Here's a breakdown of the situation and potential solutions:
The Problem:
Knox is designed to write audit logs to Ranger for topologies it creates by default (cdp-proxy, cdp-proxy-api, etc.).
You've created custom topologies (user1, topo1, topo2, app) and migrated some from HDP.
Knox only writes Ranger logs for the default cdp* topologies, not the custom ones.
Why it might not be Ranger or Solr:
You've confirmed configurations and XMLs seem correct (except for expected differences).
If Ranger or Solr were misconfigured, it would likely affect all Knox topologies, not just custom ones.
Possible Solutions:
Check Knox Logging:
Look for errors related to custom topologies in Knox logs (usually /var/log/knox/gateway/gateway.log).
Verify Ranger Topology Sync:
Ensure Ranger is configured to synchronize with all Knox topologies, including custom ones. Refer to Cloudera documentation on Knox Topology Management in CDP Private Cloud for details on this process.
Community Resources:
Search the Cloudera Community forums for "Knox not writing custom topology audit logs". There might be existing solutions or discussions relevant to your issue (https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/knox-authentication/topics/security-knox-topo...).
Cloudera Support:
If none of the above solutions work, consider contacting Cloudera Support for further assistance. They can provide deeper investigation and potential bug fixes.
I hope the information may helps you.
Created 06-05-2024 02:26 AM
Hello @Dennisleonn,
Thank you for the detailed explanation and response. Certainly helped to understand the way Knox and Ranger work together.
With respect to the issue Knox being not able to write the audit logs, I was able to get it through by changing the authorization type to "XASecurePDPKnox", which pushed Knox to use Ranger for authorizations and ultimately started writing audits to HDFS.
However, I am now stuck on next issue where, I am unable to access the service URLs from Knox as regardless of the permissions in ranger policies for respective service, access is denied. Same is seen on Ranger Admin UI as well, which confirms ranger is denying access to service UIs via custom topology.
All works okay with default (cdp-proxy) topology.
I am pretty sure something basic is missed. But unable to get hold of it. Any clue on this?
Thanks
snm1523
Created 06-07-2024 01:19 AM
UPDATE: There were configs done incorrectly in topology files due to which I was unable to access Service UIs. Fixing the topology config file helped.