Skip to main content
aleezeh
Silver ⭐️⭐️
November 29, 2022
Question

Unable to get audit logs from Apache Ranger Trino plugin

I have edited the install.properties file for the trino plugin and am using this repository https://github.com/aakashnand/trino-ranger-demo.git 

I edited the  XAAUDIT.ELASTICSEARCH.URL to my IP and XAAUDIT.ELASTICSEARCH.PORT to my port, and I am using similar configuration for apache hive plugin too and its working there but for trino I get this error when I perform a task and it tried to send audit message.

 

 

2022-11-29T09:48:32.780Z	INFO	org.apache.ranger.audit.queue.AuditBatchQueue1	stdout	INFO  - {"repoType":8,"repo":"apache_trino_onprem_8","reqUser":"testuser","evtTime":"2022-11-29 09:48:32.543","resource":"tpch/sf1/nation/comment","resType":"column","action":"select","result":1,"agent":"trino","policy":141,"enforcer":"ranger-acl","agentHost":"my-localhost-trino","logType":"RangerAudit","id":"718df890-2d9c-4641-96fe-c38fd58f8a20-37","seq_num":75,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"","policy_version":0}
2022-11-29T09:48:33.471Z INFO org.apache.ranger.audit.queue.AuditBatchQueue0 stdout ERROR - Can't connect to ElasticSearch server: User:elasticsearch, http://192.168.100.82:8091/ranger_auditsjava.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1891)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1645)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572)
at org.elasticsearch.client.IndicesClient.open(IndicesClient.java:614)
at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.newClient(ElasticSearchAuditDestination.java:253)
at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.getClient(ElasticSearchAuditDestination.java:184)
at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:118)
at org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
at java.base/java.lang.Thread.run(Thread.java:833)

2022-11-29T09:48:33.471Z INFO org.apache.ranger.audit.queue.AuditBatchQueue0 stdout WARN - failed to log audit event: {"repoType":8,"repo":"apache_trino_onprem_8","reqUser":"testuser","evtTime":"2022-11-29 09:48:31.428","resource":"lower","resType":"function","action":"execute","result":0,"agent":"trino","policy":-1,"enforcer":"ranger-acl","agentHost":"my-localhost-trino","logType":"RangerAudit","id":"718df890-2d9c-4641-96fe-c38fd58f8a20-19","seq_num":39,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":""}, errorMessage=
2022-11-29T09:48:33.476Z INFO org.apache.ranger.audit.queue.AuditBatchQueue0 stdout WARN - Log failure count: 19 in past 28:23.520 minutes; 20 during process lifetime
2022-11-29T09:48:33.669Z INFO PolicyRefresher(serviceName=apache_trino_onprem_8)-169 stdout ERROR - Error getting Roles; service not found. secureMode=false, user=root (auth:SIMPLE), response=404, serviceName=apache_trino_onprem_8, lastKnownRoleVersion=-1, lastActivationTimeInMillis=1669713508318

 

 



I also 
can't find where the curl commands are, e.g for hive I can see the exact curl command it is sending along with the payload in hive.log but for trino I can't see any curl command in logs. 

7 replies

smruti
Gold ⭐️⭐️⭐️
Gold ⭐️⭐️⭐️
November 30, 2022

@aleezeh We seem to be missing the jar for the class org.elasticsearch.ElasticsearchStatusException

Please make sure that you have elasticsearch-7.6.0.jar is present in the classpath. This jar comes bundled in ranger-${RANGER_VERSION}-admin.tar.gz. 

Translate
aleezeh
aleezehAuthor
Silver ⭐️⭐️
December 1, 2022

hey, I'm not using ranger-admin I'm using the <es url>/_bulk call to get the audits which works up until trino 365, this error is only visible in the newer trino versions

Translate
smruti
Gold ⭐️⭐️⭐️
Gold ⭐️⭐️⭐️
December 1, 2022

@aleezeh I guess you could reach out to Trino support for their inputs. But it looks like we are missing a dependent elasticsearch jar in the classpath.

Translate
aleezeh
aleezehAuthor
Silver ⭐️⭐️
January 10, 2023

hey! so the problem was that i wasnt handling the open connection API call for elastic search due to which i was facing the error 

Translate
aleezeh
aleezehAuthor
Silver ⭐️⭐️
May 2, 2023

I was redirecting the es audit logs to get the audit logs on an endpoint I had made for testing and the issue was the i wasnt handling the open connection es API call due to which the plugin wasnt able to establish an es connection, 

/ranger_audits/_open

This is the API end point and this needs to send some response so es audit connection can be established 

Translate