question Re: Ranger audit to Solr problem in Support Questions

Ranger audit to Solr problem

frank93 — Mon, 19 Aug 2019 09:44:36 GMT

Hi guys,

I upgraded HDP from 2.3 to 2.5 (and Ambari to 2.4) using Rolling Upgrade. I want to use Solr as Ranger audit type. To do that I installed Ambari-infra and configured Ranger. I can access Solr UI using hostname:8886. Ambari created ranger audits core during restart, but in ranger UI in audit tab I got an error: "Unable to connect to Audit store !!". Below is my configuration and xa_portal.log.

The command "/usr/lib/ambari-infra-solr-client/solrCloudCli.sh --zookeeper-connect-string hadoop3.locald:2181,hadoop2.locald:2181,hadoop1.locald:2181/infra-solr --create-collection --collection ranger_audits --config-set ranger_audits --shards 1 --replication 1 --max-shards 1 --retry 5 --interval 10 --no-sharding" successfully creates a core.

I am using 1 Solr server

xa-portal.txt

Re: Ranger audit to Solr problem

frank93 — Thu, 13 Oct 2016 21:07:56 GMT

In solr log I found that:

"ERROR [c:ranger_audits s:shard1 r:core_node1 x:ranger_audits] org.apache.solr.common.SolrException (SolrException.java:148) - org.apache.solr.common.SolrException: No registered leader was found after waiting for 4000ms , collection: ranger_audits slice: shard1"

and

ERROR [c:ranger_audits s:shard1 r:core_node1 x:ranger_audits] org.apache.solr.common.SolrException (SolrException.java:148) - org.apache.solr.common.SolrException: sort param field can't be found: evtTime

I did not configure any username/password for Solr. However in Ambari the values for that must not be blank. I got a WARN in xa_portal.log that "zkClient received AuthFailed"

Re: Ranger audit to Solr problem

dsharma — Thu, 13 Oct 2016 21:27:30 GMT

there seems some issue with solr and zk communitcation, can you please try restarting the solr server and check again

Re: Ranger audit to Solr problem

frank93 — Thu, 13 Oct 2016 21:33:00 GMT

thank you for quick answer, but nothing happened after restart

I think the connection with zk is ok because it uses Zk to create a core/collection

Re: Ranger audit to Solr problem

vperiasamy — Thu, 13 Oct 2016 22:06:14 GMT

If using Ambari Infra Solr, collection will be automatically created when you restart ranger from ambari. There is no need to create explicitly. Can you clean up that, clear the cookies in your browser, restart ranger service and try again?

Re: Ranger audit to Solr problem

frank93 — Mon, 19 Aug 2019 09:44:22 GMT

After restarting ranger, the collection is created automatically, but with name "ranger_audits_shard1_replica1". That is why I created (copied) new collection.

Re: Ranger audit to Solr problem

jstraub — Thu, 13 Oct 2016 22:55:58 GMT

@Edgar Daeds That is actually correct. Ranger is creating a solr collection, but you are looking at a single Shard of that collection on the UI. If you open another Ambari Infra Instance UI you will see that the URL changes to ..../ranger_audits_shard2_replica1....

Is this a kerberized environment?

Can you copy-paste the ranger.audit.solr.zookeepers configuration value?

Re: Ranger audit to Solr problem

frank93 — Fri, 14 Oct 2016 01:13:46 GMT

Och, ok, thank you. The full value is: "hadoop1.locald:2181,hadoop2.locald:2181,hadoop3.locald:2181/infra-solr".

It is not a kerberized environment

Re: Ranger audit to Solr problem

frank93 — Fri, 14 Oct 2016 02:01:36 GMT

Please check my xa_portal.log. The problem may be that query.

Re: Ranger audit to Solr problem

jstraub — Fri, 14 Oct 2016 11:25:17 GMT

Hi @Edgar Daeds

I looked at your log file and it seems that your solr schema is broken or not valid.

at http://myhostname:8886/solr/ranger_audits: sort param field can't be found: evtTime, retry

Could you please delete the collection and its configure. Afterwards, let Ranger re-create the Collection and its configuration.

Delete Collections: https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6

Delete Configuration (delete the collection first!):

1.Log into ZK

zookeeper-client -server <zk server & port>

2.Check what configurations are available

ls /infra-solr/configs

3.Delete configurations related to ranger audits (including the ones you have created). For example:

rmr /infra-solr/configs/ranger_audits

Now let Ranger re-create the Audit collection

🙂

Re: Ranger audit to Solr problem

frank93 — Fri, 14 Oct 2016 13:58:16 GMT

@Jonas Straub

It works! Thank you. I tried to delete it before but not using Zookeeper, just Solr admin and then I deleted my core directory.

Where does Solr store ranger audit? Locally in /opt/ambari_infra_solr/data/ranger_audits_shard1_replica1/data I see the data is getting larger and then smaller.

Re: Ranger audit to Solr problem

jstraub — Fri, 14 Oct 2016 17:48:29 GMT

Awesome! Glad it worked 🙂

Yes, the solrconfig for ranger audits is configured to use local storage. You can check the solrconfig.xml (either through zookeeper or Solr UI -> Select Collection in dropdown -> Files -> solrconfig.xml) for the exact path.

Re: Ranger audit to Solr problem

erkansirin78 — Fri, 13 Jul 2018 14:46:20 GMT

No ranger_audits

[zk: hadooptest01:2181(CONNECTED) 0] ls /infra-solr/configs

[atlas_configs]