question Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException in Support Questions

Unable to get audit logs from Apache Ranger Trino plugin

aleezeh — Thu, 01 Dec 2022 17:18:26 GMT

I have edited the install.properties file for the trino plugin and am using this repository https://github.com/aakashnand/trino-ranger-demo.git

I edited the XAAUDIT.ELASTICSEARCH.URL to my IP and XAAUDIT.ELASTICSEARCH.PORT to my port, and I am using similar configuration for apache hive plugin too and its working there but for trino I get this error when I perform a task and it tried to send audit message.

2022-11-29T09:48:32.780Z INFO org.apache.ranger.audit.queue.AuditBatchQueue1 stdout INFO - {"repoType":8,"repo":"apache_trino_onprem_8","reqUser":"testuser","evtTime":"2022-11-29 09:48:32.543","resource":"tpch/sf1/nation/comment","resType":"column","action":"select","result":1,"agent":"trino","policy":141,"enforcer":"ranger-acl","agentHost":"my-localhost-trino","logType":"RangerAudit","id":"718df890-2d9c-4641-96fe-c38fd58f8a20-37","seq_num":75,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"","policy_version":0} 2022-11-29T09:48:33.471Z INFO org.apache.ranger.audit.queue.AuditBatchQueue0 stdout ERROR - Can't connect to ElasticSearch server: User:elasticsearch, http://192.168.100.82:8091/ranger_auditsjava.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1891) at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1645) at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1602) at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1572) at org.elasticsearch.client.IndicesClient.open(IndicesClient.java:614) at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.newClient(ElasticSearchAuditDestination.java:253) at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.getClient(ElasticSearchAuditDestination.java:184) at org.apache.ranger.audit.destination.ElasticSearchAuditDestination.log(ElasticSearchAuditDestination.java:118) at org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309) at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215) at java.base/java.lang.Thread.run(Thread.java:833) 2022-11-29T09:48:33.471Z INFO org.apache.ranger.audit.queue.AuditBatchQueue0 stdout WARN - failed to log audit event: {"repoType":8,"repo":"apache_trino_onprem_8","reqUser":"testuser","evtTime":"2022-11-29 09:48:31.428","resource":"lower","resType":"function","action":"execute","result":0,"agent":"trino","policy":-1,"enforcer":"ranger-acl","agentHost":"my-localhost-trino","logType":"RangerAudit","id":"718df890-2d9c-4641-96fe-c38fd58f8a20-19","seq_num":39,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":""}, errorMessage= 2022-11-29T09:48:33.476Z INFO org.apache.ranger.audit.queue.AuditBatchQueue0 stdout WARN - Log failure count: 19 in past 28:23.520 minutes; 20 during process lifetime 2022-11-29T09:48:33.669Z INFO PolicyRefresher(serviceName=apache_trino_onprem_8)-169 stdout ERROR - Error getting Roles; service not found. secureMode=false, user=root (auth:SIMPLE), response=404, serviceName=apache_trino_onprem_8, lastKnownRoleVersion=-1, lastActivationTimeInMillis=1669713508318

I also can't find where the curl commands are, e.g for hive I can see the exact curl command it is sending along with the payload in hive.log but for trino I can't see any curl command in logs.

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

smruti — Wed, 30 Nov 2022 11:49:29 GMT

@aleezeh We seem to be missing the jar for the class org.elasticsearch.ElasticsearchStatusException

Please make sure that you have elasticsearch-7.6.0.jar is present in the classpath. This jar comes bundled in ranger-${RANGER_VERSION}-admin.tar.gz.

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

aleezeh — Thu, 01 Dec 2022 08:00:35 GMT

hey, I'm not using ranger-admin I'm using the <es url>/_bulk call to get the audits which works up until trino 365, this error is only visible in the newer trino versions

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

aleezeh — Thu, 01 Dec 2022 08:01:28 GMT

hey, I'm not using ranger-admin I'm using the <es url>/_bulk call to get the audits which works up until trino 365, this error is only visible in the newer trino versions

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

smruti — Thu, 01 Dec 2022 09:01:14 GMT

@aleezeh I guess you could reach out to Trino support for their inputs. But it looks like we are missing a dependent elasticsearch jar in the classpath.

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

aleezeh — Tue, 10 Jan 2023 22:47:20 GMT

hey! so the problem was that i wasnt handling the open connection API call for elastic search due to which i was facing the error

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

jtvm — Tue, 02 May 2023 08:04:07 GMT

Hi @aleezeh ,

I'm facing similar problems. Can you elaborate how do you solve the problem?

Thanks,

Re: Unable to get audit logs from Apache Ranger Trino plugin - java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.ElasticsearchStatusException

aleezeh — Tue, 02 May 2023 08:08:41 GMT

I was redirecting the es audit logs to get the audit logs on an endpoint I had made for testing and the issue was the i wasnt handling the open connection es API call due to which the plugin wasnt able to establish an es connection,

/ranger_audits/_open

This is the API end point and this needs to send some response so es audit connection can be established