Support Questions
Find answers, ask questions, and share your expertise

Ranger Audits on External Kerebrized SolrCloud

Rising Star

Hello ,

I am trying to push ranger audits to external kerberized solr cloud . When I configure Ranger to audit to external solrcloud which is kerebrized , I get an "401 authentication error" .

I am able to create collections using a solr rest api's , so I feel configs are good from solr end . When Ranger is restarted , I see it uses a ambari-infra-solr-client to talk to the external solr cluster and fails giving 401 authentication error as the client is unable to authenticate it self .

Running the same script and adding "-jf <path-to-jaas-conf>" enables me to create collection from the command line .

I am trying to see , how can I achive this via ambari . Am I missing any configs on Ranger end which would flag ambari-infra to pass the jaas conf file ?

Below is the output when I trigger a restart from ambari:

14163-screen-shot-2017-03-28-at-63145-pm.png

14164-screen-shot-2017-03-28-at-63230-pm.png

Running the command from command line and passing the jaas conf as a -jf parameter runs fine and the collection gets created .

[root@hdl-n1 ~]# /usr/lib/ambari-infra-solr-client/solrCloudCli.sh --zookeeper-connect-string hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr --create-collection -jf /etc/ambari-infra-solr/conf/infra_solr_jaas.conf --collection ranger_audits3 --config-set ranger_audits --shards 1 --replication 1 --max-shards 1 --retry 5 --interval 10 --no-sharding
Using default ZkCredentialsProvider
Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
Client environment:host.name=hdl-n1.zalonilabs.com
Client environment:java.version=1.7.0_67
Client environment:java.vendor=Oracle Corporation
Client environment:java.home=/usr/java/jdk1.7.0_67/jre
Client environment:java.class.path=/usr/lib/ambari-infra-solr-client:/usr/lib/ambari-infra-solr-client/libs/commons-io-2.1.jar:/usr/lib/ambari-infra-solr-client/libs/junit-4.10.jar:/usr/lib/ambari-infra-solr-client/libs/woodstox-core-asl-4.4.1.jar:/usr/lib/ambari-infra-solr-client/libs/slf4j-log4j12-1.7.2.jar:/usr/lib/ambari-infra-solr-client/libs/commons-lang-2.5.jar:/usr/lib/ambari-infra-solr-client/libs/jackson-mapper-asl-1.9.13.jar:/usr/lib/ambari-infra-solr-client/libs/stax2-api-3.1.4.jar:/usr/lib/ambari-infra-solr-client/libs/log4j-1.2.17.jar:/usr/lib/ambari-infra-solr-client/libs/noggit-0.6.jar:/usr/lib/ambari-infra-solr-client/libs/objenesis-2.2.jar:/usr/lib/ambari-infra-solr-client/libs/slf4j-api-1.7.2.jar:/usr/lib/ambari-infra-solr-client/libs/httpcore-4.4.1.jar:/usr/lib/ambari-infra-solr-client/libs/easymock-3.4.jar:/usr/lib/ambari-infra-solr-client/libs/httpclient-4.4.1.jar:/usr/lib/ambari-infra-solr-client/libs/commons-cli-1.3.1.jar:/usr/lib/ambari-infra-solr-client/libs/solr-solrj-5.5.2.jar:/usr/lib/ambari-infra-solr-client/libs/jackson-core-asl-1.9.9.jar:/usr/lib/ambari-infra-solr-client/libs/ambari-logsearch-solr-client-2.4.2.0.136.jar:/usr/lib/ambari-infra-solr-client/libs/zookeeper-3.4.6.jar:/usr/lib/ambari-infra-solr-client/libs/hamcrest-core-1.1.jar:/usr/lib/ambari-infra-solr-client/libs/httpmime-4.4.1.jar:/usr/lib/ambari-infra-solr-client/libs/jcl-over-slf4j-1.7.7.jar:/usr/lib/ambari-infra-solr-client/libs/commons-codec-1.8.jar
Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Client environment:java.io.tmpdir=/tmp
Client environment:java.compiler=<NA>
Client environment:os.name=Linux
Client environment:os.arch=amd64
Client environment:os.version=2.6.32-642.el6.x86_64
Client environment:user.name=root
Client environment:user.home=/root
Client environment:user.dir=/root
Initiating client connection, connectString=hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr sessionTimeout=15000 watcher=org.apache.solr.common.cloud.SolrZkClient$3@7a04c4aa
Waiting for client to connect to ZooKeeper
successfully logged in.
TGT refresh thread started.
Client will use GSSAPI as SASL mechanism.
TGT valid starting at:        Sat Mar 25 13:29:52 EDT 2017
TGT expires:                  Sun Mar 26 13:29:52 EDT 2017
TGT refresh sleeping until: Sun Mar 26 09:01:47 EDT 2017
Opening socket connection to server hdl-n3.zalonilabs.com/10.11.13.168:2181. Will attempt to SASL-authenticate using Login Context section 'Client'
Socket connection established to hdl-n3.zalonilabs.com/10.11.13.168:2181, initiating session
Session establishment complete on server hdl-n3.zalonilabs.com/10.11.13.168:2181, sessionid = 0x35af687cfb80050, negotiated timeout = 15000
Watcher org.apache.solr.common.cloud.ConnectionManager@7a53c84a name:ZooKeeperConnection Watcher:hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
Client is connected to ZooKeeper
Using default ZkACLProvider
Watcher org.apache.solr.common.cloud.ConnectionManager@7a53c84a name:ZooKeeperConnection Watcher:hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr got event WatchedEvent state:SaslAuthenticated type:None path:null path:null type:None
Setting up SPNego auth with config: /etc/ambari-infra-solr/conf/infra_solr_jaas.conf
Using default ZkCredentialsProvider
Initiating client connection, connectString=hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$3@3b0f2591
Waiting for client to connect to ZooKeeper
Client will use GSSAPI as SASL mechanism.
Opening socket connection to server hdl-n1.zalonilabs.com/10.11.13.166:2181. Will attempt to SASL-authenticate using Login Context section 'Client'
Socket connection established to hdl-n1.zalonilabs.com/10.11.13.166:2181, initiating session
Session establishment complete on server hdl-n1.zalonilabs.com/10.11.13.166:2181, sessionid = 0x15af687cf9e004e, negotiated timeout = 10000
Watcher org.apache.solr.common.cloud.ConnectionManager@54274d27 name:ZooKeeperConnection Watcher:hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
Client is connected to ZooKeeper
Using default ZkACLProvider
Updating cluster state from ZooKeeper...
Watcher org.apache.solr.common.cloud.ConnectionManager@54274d27 name:ZooKeeperConnection Watcher:hdl-n3.zalonilabs.com:2181,hdl-n2.zalonilabs.com:2181,hdl-n1.zalonilabs.com:2181/solr got event WatchedEvent state:SaslAuthenticated type:None path:null path:null type:None
A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
Collection 'ranger_audits3' created.
Return code: 0

Thanks,

Jagdish

4 REPLIES 4

Rising Star

@Jonas Straub any input on the above .

New Contributor

@Jonas Straub which document are you following? Can you rehash your setup configuration using this document.

Rising Star

Followed the same document "Configure Solrcloud Deployment for Kerberos" .

I already had a solr cloud deployed under /opt/lucidworks/... and want tthe existing solrcloud infrastructure to store Ranger audits . I happen to kerebrize the solrcloud and same steps layed out in the document was followed .

Please note I am able to create collections using solr REST API's & also using ambari-infra-solr-client from command line by passing jaas file , but not able to do so when ranger does all the pre checks while restarting.

if using external solr cloud, you are expected to create the collection yourself and modify ranger audit config to use external solr (i.e. this should have updated zookeeper setting with the correct znode). I am assuming these steps are performed. Can you post your configs and any other errors you see from Ambari?