Support Questions

Find answers, ask questions, and share your expertise

Ranger Audit stopped working after server reboot

avatar
Super Collaborator

Hello

We use Ranger for auditing Hive. Yesterday at 15:20 after restarting the Server that runs Ambari (Ambari audits to the default mysql which is on the same server) the auditing has stopped working. Ranger is up & running, mysql is also running, plugins are working and synced with Hiveserver2. Yet no data is being audited to he db since the reboot of the server. I've queried the xa_ranger_audit table and it shows entries up until the server reboot.

Clearly something imperative is not running.

Any suggestions are appreciated !

9856-snap-2016-11-29-at-110830.png

9857-snap-2016-11-29-at-110847.png

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Found the problem.

Investigating the hiveserver2.log showed that rangerlogger failed to flush data to the db due to permission problem (Internal Exception: java.sql.SQLException: Access denied for user 'rangerlogger'@'<server>' (using password: YES) Error Code: 1045)

After granting the permissions it flushed the data successfully.

Now i just need to figure out what caused the loss of permissions and how the hell it is related to the server reboot....

Thanks for all your help !

View solution in original post

10 REPLIES 10

avatar
Super Guru

@Adi Jabkowsky

Can you check if you see any error in xa_portal.log?

Can you try disabling the plugin from configs for respective services and re-enable?

Please try to enable ranger debug -

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Menlo} span.s1 {font-variant-ligatures: no-common-ligatures}

/usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/log4j.xml

Change From:

<category name="org.apache.ranger" additivity="false">
		<priority value="info" />
		<appender-ref ref="xa_log_appender" />
</category>



Change To:
<category name="org.apache.ranger" additivity="false">
		<priority value="debug" />
		<appender-ref ref="xa_log_appender" />
</category>


Restart Ranger admin and check the xa_portal logs again.

avatar
Super Collaborator

@Sagar Shimpi Thank you for replying. I wanted to add that login sessions are being audited. It's just the Access information (Hive access) ism't audited. I enabled debug as per your recommendation and restarted ranger but haven't found any errors regarding audit or connection to db. There is an LDAP error: DEBUG org.apache.ranger.security.handler.RangerAuthenticationProvider (RangerAuthenticationProvider.java:280) - AD Authentication Failed: org.springframework.security.authentication.BadCredentialsException: Bad credentials

But it doesn't say which user and i'm not sure it is related. Our Hive is configured for LDAP auth so maybe it is related.

avatar
Super Collaborator

Hi @Adi Jabkowsky

please check properties:

- xasecure.audit.destination.db.user

- xasecure.audit.destination.db.password

- xasecure.audit.destination.db.jdbc.url

- xasecure.audit.destination.db.jdbc.driver

and whether "Audit to DB" is checked in Ambari -> Hive -> Configs -> Advanced -> Advanced ranger-hive-audit. Try to type your own values instead of {{variable}}.

avatar
Super Collaborator

@Edgar Daeds

Thank you for replying.

Audit to DB is checked in Ambari. I also unchecked, saved and restarted, then checked again, saved and restarted - but still problem remains.

Regarding using own values instead of variables - i'm not sure exactly what to enter and again - until the reboot it worked flawlessly.

avatar
Super Collaborator

@Adi Jabkowsky

I had a similar problem to yours once (works till reboot). I typed the values of these parameters:

- xasecure.audit.destination.db.user

- xasecure.audit.destination.db.password

as "plain text" and it worked for me. First one is a MySQL Username which has access to ranger_audit database and second parameter is his password.

avatar
Super Collaborator

@Edgar Daeds

I've changed those settings to clear text and restarted hive + Ranger but problem remains unfortunately.

avatar
Super Collaborator

One more thing to add: we restarted the server after installing and configuring sssd on the server for group mapping.

avatar
Super Guru

If ranger is integrated with ldap then you need to revisit the config.

avatar
Super Collaborator

@Sagar Shimpi

Can you be more specific ? The Ranger authentication is LDAP, meaning in order to login to it's UI you can only login using your Active Directory credentials. This works without any problems.