question Re: Ranger Audit stopped working after server reboot in Archives of Support Questions (Read Only)

Ranger Audit stopped working after server reboot

Adija1 — Mon, 19 Aug 2019 09:59:16 GMT

Hello

We use Ranger for auditing Hive. Yesterday at 15:20 after restarting the Server that runs Ambari (Ambari audits to the default mysql which is on the same server) the auditing has stopped working. Ranger is up & running, mysql is also running, plugins are working and synced with Hiveserver2. Yet no data is being audited to he db since the reboot of the server. I've queried the xa_ranger_audit table and it shows entries up until the server reboot.

Clearly something imperative is not running.

Any suggestions are appreciated !

Re: Ranger Audit stopped working after server reboot

sshimpi — Tue, 29 Nov 2016 17:19:12 GMT

@Adi Jabkowsky

Can you check if you see any error in xa_portal.log?

Can you try disabling the plugin from configs for respective services and re-enable?

Please try to enable ranger debug -

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Menlo} span.s1 {font-variant-ligatures: no-common-ligatures}

/usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/log4j.xml

Change From:

<category name="org.apache.ranger" additivity="false">
		<priority value="info" />
		<appender-ref ref="xa_log_appender" />
</category>



Change To:
<category name="org.apache.ranger" additivity="false">
		<priority value="debug" />
		<appender-ref ref="xa_log_appender" />
</category>

Restart Ranger admin and check the xa_portal logs again.

Re: Ranger Audit stopped working after server reboot

frank93 — Tue, 29 Nov 2016 17:23:39 GMT

Hi @Adi Jabkowsky

please check properties:

- xasecure.audit.destination.db.user

- xasecure.audit.destination.db.password

- xasecure.audit.destination.db.jdbc.url

- xasecure.audit.destination.db.jdbc.driver

and whether "Audit to DB" is checked in Ambari -> Hive -> Configs -> Advanced -> Advanced ranger-hive-audit. Try to type your own values instead of {{variable}}.

Re: Ranger Audit stopped working after server reboot

Adija1 — Tue, 29 Nov 2016 20:13:06 GMT

@Sagar Shimpi Thank you for replying. I wanted to add that login sessions are being audited. It's just the Access information (Hive access) ism't audited. I enabled debug as per your recommendation and restarted ranger but haven't found any errors regarding audit or connection to db. There is an LDAP error: DEBUG org.apache.ranger.security.handler.RangerAuthenticationProvider (RangerAuthenticationProvider.java:280) - AD Authentication Failed: org.springframework.security.authentication.BadCredentialsException: Bad credentials

But it doesn't say which user and i'm not sure it is related. Our Hive is configured for LDAP auth so maybe it is related.

Re: Ranger Audit stopped working after server reboot

Adija1 — Tue, 29 Nov 2016 20:15:42 GMT

@Edgar Daeds

Thank you for replying.

Audit to DB is checked in Ambari. I also unchecked, saved and restarted, then checked again, saved and restarted - but still problem remains.

Regarding using own values instead of variables - i'm not sure exactly what to enter and again - until the reboot it worked flawlessly.

Re: Ranger Audit stopped working after server reboot

frank93 — Tue, 29 Nov 2016 20:20:52 GMT

@Adi Jabkowsky

I had a similar problem to yours once (works till reboot). I typed the values of these parameters:

- xasecure.audit.destination.db.user

- xasecure.audit.destination.db.password

as "plain text" and it worked for me. First one is a MySQL Username which has access to ranger_audit database and second parameter is his password.

Re: Ranger Audit stopped working after server reboot

Adija1 — Tue, 29 Nov 2016 20:23:14 GMT

One more thing to add: we restarted the server after installing and configuring sssd on the server for group mapping.

Re: Ranger Audit stopped working after server reboot

sshimpi — Tue, 29 Nov 2016 21:46:48 GMT

If ranger is integrated with ldap then you need to revisit the config.

Re: Ranger Audit stopped working after server reboot

Adija1 — Tue, 29 Nov 2016 23:43:44 GMT

@Sagar Shimpi

Can you be more specific ? The Ranger authentication is LDAP, meaning in order to login to it's UI you can only login using your Active Directory credentials. This works without any problems.

Re: Ranger Audit stopped working after server reboot

Adija1 — Tue, 29 Nov 2016 23:58:06 GMT

@Edgar Daeds

I've changed those settings to clear text and restarted hive + Ranger but problem remains unfortunately.

Re: Ranger Audit stopped working after server reboot

Adija1 — Wed, 30 Nov 2016 01:40:23 GMT

Found the problem.

Investigating the hiveserver2.log showed that rangerlogger failed to flush data to the db due to permission problem (Internal Exception: java.sql.SQLException: Access denied for user 'rangerlogger'@'<server>' (using password: YES) Error Code: 1045)

After granting the permissions it flushed the data successfully.

Now i just need to figure out what caused the loss of permissions and how the hell it is related to the server reboot....

Thanks for all your help !