I have a requirement where external user should be able to get admin rights on Ranger admin UI. I followed this article to configure and it works fine when I use LDAP as directory. However when I tried the same with Active directory as my directory server, it does not work. In both of the cases, all other configurations remain similar (i.e. my Ranger is syncing users from Active directory as well as I'm able to log in using users from Active directory). But somehow the admin user configuration does not work for me. Please can someone help? ... View more

I'm using setup as follows: 1. My Ranger is authenticated using knox via LDAP 2. Ranger is synced with LDAP - I can see users and groups in Ranger 3. Additionally I added a group (even tried with user) from LDAP and mapped with each permission in Ranger Permissions screen (by first disabling knox authentication, using admin user to login to Ranger GUI and then again enabling it). However when I log in using an LDAP user who is mapped with all Ranger permissions; ranger still does not allow full rights to user (e.g. + icon for creating a repository is missing). Also this user does not see policies created by ranger user 'admin' Since the access to ranger user named 'admin' is not available when I configure Ranger to do sso authentication with knox; I have no way to do administrative tasks in ranger, since none of the LDAP users (authenticated to ranger admin via knox-ldap) will have permission equivalant to ranger's internal user named 'admin'. How can I configure Ranger to allow one group/user from LDAP to act as Ranger Admin's admin? ... View more

What I'm trying to achieve: Integrate Ranger and NiFi so that nifi authorization can be governed by Ranger. I want Ranger audit logs to be flushed to HDFS as they generate (or with a minimum delay). I do not really need nifi in secure mode at this point, but since nifi on http did not seem to be use ranger policies, I ended up making it https enabled. I do not want the audit logs to go to Solr - it should just go to HDFS. My setup: HDP 2.6 sandbox on one centos instance & nifi 4 on another centos instance My ranger is not ssl enabled i.e. can be accessed by http My nifi is ssl enabled. Note: My nifi is standalone i.e. is not administered/installed using Ambari. I also configured ranger as an authorizer in NiFi. I created a generous policy in Ranger granting access. initially I did have Solr also as audit destination (apart from HDFS) but later I changed it to false in "ranger-nifi-audit.xml". I do have hdfs as a destination configured in "ranger-nifi-audit.xml". My observations: Nifi is getting governed by Ranger policies. the ranger plugin also audits logs in spool directory But audit log is not flushed to HDFS - even after wait of couple of hours. Somehow I go impression that reducing "xasecure.audit.destination.hdfs.file.rollover.sec" will get faster updates to HDFS; which I found is wrong. It simply ends up creating lots of empty files. I checked spool directory and it does have logs captured. I also see messages like following in the nifi-app.log periodically: 2017-11-06 18:59:56,901 INFO [org.apache.ranger.audit.queue.AuditBatchQueue0] o.a.r.audit.provider.BaseAuditHandler Audit Status Log: name=nifi.async.batch.hdfs, interval=01:00.011 minutes, events=10, succcessCount=10, totalEvents=125, totalSuccessCount=1252017-11-06 18:59:56,902 INFO [org.apache.ranger.audit.queue.AuditBatchQueue0] o.a.r.a.destination.HDFSAuditDestination Flushing HDFS audit. Event Size:5 But even after waiting for couple of hours, I do not see any audit log dumped into HDFS. Just empty file sitting there. I do not see any errors about connection to HDFS in nifi-app.log edit: My ranger-nifi-audit.xml is as below. Recently added log4j entry as per https://community.hortonworks.com/questions/118294/can-nifi-ranger-plugin-audit-log-to-hdfs.html But even that does not seem to capture logs through log4j. <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration xmlns:xi="http://www.w3.org/2001/XInclude"> <property> <name>xasecure.audit.is.enabled</name> <value>true</value> </property> <property> <name>xasecure.audit.destination.solr</name> <value>false</value> </property> <property> <name>xasecure.audit.destination.solr.batch.filespool.dir</name> <value>/tmp/audit/solr/spool</value> </property> <property> <name>xasecure.audit.destination.solr.urls</name> <value>http://192.168.1.12:6083/solr/ranger_audits</value> </property> <property> <name>xasecure.audit.destination.hdfs</name> <value>true</value> </property> <property> <name>xasecure.audit.destination.hdfs.dir</name> <value>hdfs://sandbox.hortonworks.com:8020/ranger/audit</value> </property> <property> <name>xasecure.audit.destination.hdfs.subdir</name> <value>%app-type%/%time:yyyyMMdd%</value> </property> <property> <name>xasecure.audit.destination.hdfs.filename.format</name> <value>%app-type%_ranger_audit_%hostname%.log</value> </property> <property> <name>xasecure.audit.destination.hdfs.file.rollover.sec</name> <value>86400</value> </property> <property> <name>xasecure.audit.destination.hdfs.batch.filespool.dir</name> <value>/tmp/audit/hdfs/spool</value> </property> <property> <name>xasecure.audit.destination.hdfs.batch.interval.ms</name> <value>3000</value> </property> <property> <name>xasecure.audit.hdfs.config.destination.flush.interval.seconds</name> <value>900</value> </property> <property> <name>xasecure.audit.hdfs.is.enabled</name> <value>true</value> </property> <property> <name>xasecure.audit.hdfs.is.async</name> <value>true</value> </property> <property> <name>xasecure.audit.hdfs.async.max.queue.size</name> <value>1048576</value> </property> <property> <name>xasecure.audit.hdfs.async.max.flush.interval.ms</name> <value>30000</value> </property> <property> <name>xasecure.audit.log4j.is.enabled</name> <value>true</value> </property> <property> <name>xasecure.audit.log4j.is.async</name> <value>true</value> </property> <property> <name>xasecure.audit.destination.log4j.logger</name> <value>ranger.audit</value> </property> </configuration> Question: 1) What I may be doing wrong that though the empty file is generated, no logs added into it? Also what do I need to do to instruct nifi-ranger plugin to flush logs to hdfs with a minimum/no delay? 2) Is it possible to use unsecured nifi to connect to unsecured Ranger and still work as normal? Requesting your help, been stuck on this for quite log and not finding any articles/references addressing issue and questions I'm facing. Note: My current setup is experimental only (with a view to take it forward) at this point. ... View more

My OS: Centos 6.9 32 bit JDK: 1.8 I have already made cluster of 5 instanced (on virtualbox) ready including passwordless ssh, iptables turned off, ntpd started etc. Because Ambari's latest version was running into problems (another question raised on the community), I chose version 2.0.0 using which I was able to successfully install ambari-server and also done setup. Also installed correct (based on combination given in installation doc) repo files for hdp and ambari on all hosts. When I started ambari-server, access through GUI and reached to registration page (i.e. confim hosts); it failed for all hosts. Failure message (part of it) is: scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py host=hw-pc-namenode.pc.com, exitcode=0 Command end time 2017-09-14 13:53:59 ========================== Running setup agent script... ========================== Command start time 2017-09-14 13:53:59 /bin/sh: /usr/sbin/ambari-agent: No such file or directory Error: No matching Packages to list Error: No matching Packages to list ('', None) Connection to hw-pc-namenode.pc.com closed. SSH command execution finished host=hw-pc-namenode.pc.com, exitcode=1 Command end time 2017-09-14 13:54:04 ERROR: Bootstrap of host hw-pc-namenode.pc.com fails because previous action finished with non-zero exit code (1) ERROR MESSAGE: tcgetattr: Invalid argument Connection to hw-pc-namenode.pc.com closed. STDOUT: /bin/sh: /usr/sbin/ambari-agent: No such file or directory Error: No matching Packages to list Error: No matching Packages to list ('', None) Connection to hw-pc-namenode.pc.com closed. Since the failure is about finding ambari-agent file, I tried manually install using "yum install ambari-agent*" (Also tried yum clean all prior to this). However it was giving message ambari-agent not found. Hence used the tarball link found in doc for Ambari 2.0.0, download the tarball and glanced through it on local machine - it does not seem to have ambari-agent rpm. tried searching google, but no use. Similarly I check few of the other versions of tarballs and found only 1.x has ambari-agent rpm. I'm not much good at linux, so this left me confused. ps: I'm not behind firewall which I did check during investigation on my last question. Can you please help me with: 1. How should I overcome this error of ambari-agent (for 2.0.0) not found? 2. Normally, How is yum supposed to work n download ambari agent when it is not found in tarball as well (just for my knowledge)? Many thanks in advance of your help. ... View more