Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Ranger Audit stopped working after server reboot

Solved Go to solution
Highlighted

Ranger Audit stopped working after server reboot

Expert Contributor

Hello

We use Ranger for auditing Hive. Yesterday at 15:20 after restarting the Server that runs Ambari (Ambari audits to the default mysql which is on the same server) the auditing has stopped working. Ranger is up & running, mysql is also running, plugins are working and synced with Hiveserver2. Yet no data is being audited to he db since the reboot of the server. I've queried the xa_ranger_audit table and it shows entries up until the server reboot.

Clearly something imperative is not running.

Any suggestions are appreciated !

9856-snap-2016-11-29-at-110830.png

9857-snap-2016-11-29-at-110847.png

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Ranger Audit stopped working after server reboot

Expert Contributor

Found the problem.

Investigating the hiveserver2.log showed that rangerlogger failed to flush data to the db due to permission problem (Internal Exception: java.sql.SQLException: Access denied for user 'rangerlogger'@'<server>' (using password: YES) Error Code: 1045)

After granting the permissions it flushed the data successfully.

Now i just need to figure out what caused the loss of permissions and how the hell it is related to the server reboot....

Thanks for all your help !

10 REPLIES 10

Re: Ranger Audit stopped working after server reboot

@Adi Jabkowsky

Can you check if you see any error in xa_portal.log?

Can you try disabling the plugin from configs for respective services and re-enable?

Please try to enable ranger debug -

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Menlo} span.s1 {font-variant-ligatures: no-common-ligatures}

/usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/log4j.xml

Change From:

<category name="org.apache.ranger" additivity="false">
		<priority value="info" />
		<appender-ref ref="xa_log_appender" />
</category>



Change To:
<category name="org.apache.ranger" additivity="false">
		<priority value="debug" />
		<appender-ref ref="xa_log_appender" />
</category>


Restart Ranger admin and check the xa_portal logs again.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

@Sagar Shimpi Thank you for replying. I wanted to add that login sessions are being audited. It's just the Access information (Hive access) ism't audited. I enabled debug as per your recommendation and restarted ranger but haven't found any errors regarding audit or connection to db. There is an LDAP error: DEBUG org.apache.ranger.security.handler.RangerAuthenticationProvider (RangerAuthenticationProvider.java:280) - AD Authentication Failed: org.springframework.security.authentication.BadCredentialsException: Bad credentials

But it doesn't say which user and i'm not sure it is related. Our Hive is configured for LDAP auth so maybe it is related.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

Hi @Adi Jabkowsky

please check properties:

- xasecure.audit.destination.db.user

- xasecure.audit.destination.db.password

- xasecure.audit.destination.db.jdbc.url

- xasecure.audit.destination.db.jdbc.driver

and whether "Audit to DB" is checked in Ambari -> Hive -> Configs -> Advanced -> Advanced ranger-hive-audit. Try to type your own values instead of {{variable}}.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

@Edgar Daeds

Thank you for replying.

Audit to DB is checked in Ambari. I also unchecked, saved and restarted, then checked again, saved and restarted - but still problem remains.

Regarding using own values instead of variables - i'm not sure exactly what to enter and again - until the reboot it worked flawlessly.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

@Adi Jabkowsky

I had a similar problem to yours once (works till reboot). I typed the values of these parameters:

- xasecure.audit.destination.db.user

- xasecure.audit.destination.db.password

as "plain text" and it worked for me. First one is a MySQL Username which has access to ranger_audit database and second parameter is his password.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

@Edgar Daeds

I've changed those settings to clear text and restarted hive + Ranger but problem remains unfortunately.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

One more thing to add: we restarted the server after installing and configuring sssd on the server for group mapping.

Re: Ranger Audit stopped working after server reboot

If ranger is integrated with ldap then you need to revisit the config.

Re: Ranger Audit stopped working after server reboot

Expert Contributor

@Sagar Shimpi

Can you be more specific ? The Ranger authentication is LDAP, meaning in order to login to it's UI you can only login using your Active Directory credentials. This works without any problems.

Don't have an account?
Coming from Hortonworks? Activate your account here