Unable to get audit logs from Apache Ranger Trino plugin

[#1]

I have edited the install.properties file for the trino plugin and am using this repository https://github.com/aakashnand/trino-ranger-demo.git 

I edited the  XAAUDIT.ELASTICSEARCH.URL to my IP and XAAUDIT.ELASTICSEARCH.PORT to my port, and I am using similar configuration for apache hive plugin too and its working there but for trino I get this error when I perform a task and it tried to send audit message.

2022-11-29T09:48:32.780Z	INFO	org.apache.ranger.audit.queue.AuditBatchQueue1	stdout	INFO  - {"repoType":8,"repo":"apache_trino_onprem_8","reqUser":"testuser","evtTime":"2022-11-29 09:48:32.543","resource":"tpch/sf1/nation/comment","resType":"column","action":"select","result":1,"agent":"trino","policy":141,"enforcer":"ranger-acl","agentHost":"my-localhost-trino","logType":"RangerAudit","id":"718df890-2d9c-4641-96fe-c38fd58f8a20-37","seq_num":75,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"","policy_version":0}
2022-11-29T09:48:33.471Z	INFO	org.apache.ranger.audit.queue.AuditBatchQueue0	stdout	ERROR - Can't connect to ElasticSearch server: User:elasticsearch, http://192.168.100.82:8091/ranger_auditsjava.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException
	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1891)
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1645)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572)
	at org.elasticsearch.client.IndicesClient.open(IndicesClient.java:614)
	at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.newClient(ElasticSearchAuditDestination.java:253)
	at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.getClient(ElasticSearchAuditDestination.java:184)
	at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:118)
	at org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
	at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
	at java.base/java.lang.Thread.run(Thread.java:833)

2022-11-29T09:48:33.471Z	INFO	org.apache.ranger.audit.queue.AuditBatchQueue0	stdout	WARN  - failed to log audit event: {"repoType":8,"repo":"apache_trino_onprem_8","reqUser":"testuser","evtTime":"2022-11-29 09:48:31.428","resource":"lower","resType":"function","action":"execute","result":0,"agent":"trino","policy":-1,"enforcer":"ranger-acl","agentHost":"my-localhost-trino","logType":"RangerAudit","id":"718df890-2d9c-4641-96fe-c38fd58f8a20-19","seq_num":39,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":""}, errorMessage=
2022-11-29T09:48:33.476Z	INFO	org.apache.ranger.audit.queue.AuditBatchQueue0	stdout	WARN  - Log failure count: 19 in past 28:23.520 minutes; 20 during process lifetime
2022-11-29T09:48:33.669Z	INFO	PolicyRefresher(serviceName=apache_trino_onprem_8)-169	stdout	ERROR - Error getting Roles; service not found. secureMode=false, user=root (auth:SIMPLE), response=404, serviceName=apache_trino_onprem_8, lastKnownRoleVersion=-1, lastActivationTimeInMillis=1669713508318

I also can't find where the curl commands are, e.g for hive I can see the exact curl command it is sending along with the payload in hive.log but for trino I can't see any curl command in logs. 

[#2]

@aleezeh We seem to be missing the jar for the class org.elasticsearch.ElasticsearchStatusException

Please make sure that you have elasticsearch-7.6.0.jar is present in the classpath. This jar comes bundled in ranger-${RANGER_VERSION}-admin.tar.gz. 

[#3]

hey, I'm not using ranger-admin I'm using the <es url>/_bulk call to get the audits which works up until trino 365, this error is only visible in the newer trino versions

[#4]

hey, I'm not using ranger-admin I'm using the <es url>/_bulk call to get the audits which works up until trino 365, this error is only visible in the newer trino versions

[#5]

@aleezeh I guess you could reach out to Trino support for their inputs. But it looks like we are missing a dependent elasticsearch jar in the classpath.

[#6]

hey! so the problem was that i wasnt handling the open connection API call for elastic search due to which i was facing the error 

[#7]

Hi @aleezeh ,

I'm facing similar problems. Can you elaborate how do you solve the problem?

Thanks,

[#8]

I was redirecting the es audit logs to get the audit logs on an endpoint I had made for testing and the issue was the i wasnt handling the open connection es API call due to which the plugin wasnt able to establish an es connection, 

/ranger_audits/_open

This is the API end point and this needs to send some response so es audit connection can be established