Member since
09-12-2017
26
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1457 | 09-13-2017 04:49 AM |
09-12-2018
11:27 PM
Hi Steven, My apologies for delay - I'd been distracted. Thanks for your response. I'm able to sync users properly in both cases - 1) when I'm syncing them from LDAP (openldap) OR 2)when I'm syncing from Active Directory. In my setup, I'm authenticating to Ranger through knox and this specific issue faced only when a)I've synced Ranger from Active Directory And b)my knox is also using the same Active Directory as directory. The issue faced is to make an Active Directory user as Ranger admin user. I followed process as per link. The link has step-by-step process to configure ldap user as Ranger admin - which for me does not work in case of Active Directory.
... View more
08-17-2018
10:58 AM
Please anybody can help on this?
... View more
08-07-2018
10:12 AM
I have a requirement where external user should be able to get admin rights on Ranger admin UI. I followed this article to configure and it works fine when I use LDAP as directory. However when I tried the same with Active directory as my directory server, it does not work. In both of the cases, all other configurations remain similar (i.e. my Ranger is syncing users from Active directory as well as I'm able to log in using users from Active directory). But somehow the admin user configuration does not work for me. Please can someone help?
... View more
- Tags:
- ranger-admin
- Security
Labels:
04-10-2018
05:51 AM
@spolavarapu Thanks for your response. I tried this in Ranger version 0.7.1 (Apache) as well as in HDP2.6.1. I will try it on HDP2.6.3 (having Ranger 0.8 I think).
... View more
03-21-2018
08:53 AM
I'm using setup as follows: 1. My Ranger is authenticated using knox via LDAP 2. Ranger is synced with LDAP - I can see users and groups in Ranger 3. Additionally I added a group (even tried with user) from LDAP and mapped with each permission in Ranger Permissions screen (by first disabling knox authentication, using admin user to login to Ranger GUI and then again enabling it). However when I log in using an LDAP user who is mapped with all Ranger permissions; ranger still does not allow full rights to user (e.g. + icon for creating a repository is missing). Also this user does not see policies created by ranger user 'admin' Since the access to ranger user named 'admin' is not available when I configure Ranger to do sso authentication with knox; I have no way to do administrative tasks in ranger, since none of the LDAP users (authenticated to ranger admin via knox-ldap) will have permission equivalant to ranger's internal user named 'admin'. How can I configure Ranger to allow one group/user from LDAP to act as Ranger Admin's admin?
... View more
Labels:
03-16-2018
08:28 AM
Check Ranger User sync config. You'd be able to configure the user (as well as group) search base and accordingly Ranger User sync will be able to pull from ou=users.
... View more
01-12-2018
10:19 AM
It's similar to session maintenance in web - post login, browser receives a cookie which it sends with every request so that it's authenticity session is maintained. So, with knox it should be possible same way (if you can manage to send cookie with each request post the first one). Kerberos is another authentication mechanism so don't think adding the 2 together will help nullify one of it.
... View more
11-22-2017
04:22 AM
I recently did installation on Centos 6 and it worked fine. I did try Ambari version ~2.2 ish and also 2.6, both work fine (few niggles like while starting ambari not finding version; but that's manageable. My previous one on Centos 7 (for which this issue is raised) never worked and I left trying. Thanks Jay, Geoffrey for your help on this issue.
... View more
11-10-2017
05:01 AM
Thanks @Ramesh Mani for your help. Yes, I mean it stores in local spool directory in proper JSON format. but unable to push it to HDFS. As no errors were logged in nifi-app.log; I did try with log level DEBUG; which shown me this error messege: "File /ranger/audit/nifi/20171106/nifi_ranger_audit_hdp2503.nodelogix.com.1.log could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.". I tried solution on this as suggested in "https://stackoverflow.com/questions/5293446/hdfs-error-could-only-be-replicated-to-0-nodes-instead-of-1", but to no avail. In fact for some strange reason, now I'm not being able to access my nifi (my nifi running on one centos 7 virtual machine on virtualbox while I was accessing it from browser on my host system).
... View more
11-08-2017
12:04 PM
@Ramesh Mani Saw your response on Ranger Audit on another thread. Please can you help, I'm stuck on this since couple of days?
... View more
11-06-2017
02:22 PM
I also tried setting value of "xasecure.audit.destination.hdfs.queue" to "None"; but still audit log is not written to HDFS.
... View more
11-06-2017
01:58 PM
What I'm trying to achieve: Integrate Ranger and NiFi so that nifi authorization can be governed by Ranger. I want Ranger audit logs to be flushed to HDFS as they generate (or with a minimum delay). I do not really need nifi in secure mode at this point, but since nifi on http did not seem to be use ranger policies, I ended up making it https enabled. I do not want the audit logs to go to Solr - it should just go to HDFS. My setup: HDP 2.6 sandbox on one centos instance & nifi 4 on another centos instance My ranger is not ssl enabled i.e. can be accessed by http My nifi is ssl enabled. Note: My nifi is standalone i.e. is not administered/installed using Ambari. I also configured ranger as an authorizer in NiFi. I created a generous policy in Ranger granting access. initially I did have Solr also as audit destination (apart from HDFS) but later I changed it to false in "ranger-nifi-audit.xml". I do have hdfs as a destination configured in "ranger-nifi-audit.xml". My observations: Nifi is getting governed by Ranger policies. the ranger plugin also audits logs in spool directory But audit log is not flushed to HDFS - even after wait of couple of hours. Somehow I go impression that reducing "xasecure.audit.destination.hdfs.file.rollover.sec" will get faster updates to HDFS; which I found is wrong. It simply ends up creating lots of empty files. I checked spool directory and it does have logs captured. I also see messages like following in the nifi-app.log periodically: 2017-11-06 18:59:56,901 INFO [org.apache.ranger.audit.queue.AuditBatchQueue0] o.a.r.audit.provider.BaseAuditHandler Audit Status Log: name=nifi.async.batch.hdfs, interval=01:00.011 minutes, events=10, succcessCount=10, totalEvents=125, totalSuccessCount=1252017-11-06 18:59:56,902 INFO [org.apache.ranger.audit.queue.AuditBatchQueue0] o.a.r.a.destination.HDFSAuditDestination Flushing HDFS audit. Event Size:5 But even after waiting for couple of hours, I do not see any audit log dumped into HDFS. Just empty file sitting there. I do not see any errors about connection to HDFS in nifi-app.log edit: My ranger-nifi-audit.xml is as below. Recently added log4j entry as per https://community.hortonworks.com/questions/118294/can-nifi-ranger-plugin-audit-log-to-hdfs.html But even that does not seem to capture logs through log4j. <?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration xmlns:xi="http://www.w3.org/2001/XInclude">
<property>
<name>xasecure.audit.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.solr</name>
<value>false</value>
</property>
<property>
<name>xasecure.audit.destination.solr.batch.filespool.dir</name>
<value>/tmp/audit/solr/spool</value>
</property>
<property>
<name>xasecure.audit.destination.solr.urls</name>
<value>http://192.168.1.12:6083/solr/ranger_audits</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.dir</name>
<value>hdfs://sandbox.hortonworks.com:8020/ranger/audit</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.subdir</name>
<value>%app-type%/%time:yyyyMMdd%</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.filename.format</name>
<value>%app-type%_ranger_audit_%hostname%.log</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.file.rollover.sec</name>
<value>86400</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.batch.filespool.dir</name>
<value>/tmp/audit/hdfs/spool</value>
</property>
<property>
<name>xasecure.audit.destination.hdfs.batch.interval.ms</name>
<value>3000</value>
</property>
<property>
<name>xasecure.audit.hdfs.config.destination.flush.interval.seconds</name>
<value>900</value>
</property>
<property>
<name>xasecure.audit.hdfs.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.hdfs.is.async</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.hdfs.async.max.queue.size</name>
<value>1048576</value>
</property>
<property>
<name>xasecure.audit.hdfs.async.max.flush.interval.ms</name>
<value>30000</value>
</property>
<property>
<name>xasecure.audit.log4j.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.log4j.is.async</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.log4j.logger</name>
<value>ranger.audit</value>
</property>
</configuration>
Question: 1) What I may be doing wrong that though the empty file is generated, no logs added into it? Also what do I need to do to instruct nifi-ranger plugin to flush logs to hdfs with a minimum/no delay? 2) Is it possible to use unsecured nifi to connect to unsecured Ranger and still work as normal? Requesting your help, been stuck on this for quite log and not finding any articles/references addressing issue and questions I'm facing. Note: My current setup is experimental only (with a view to take it forward) at this point.
... View more
Labels:
09-26-2017
05:38 AM
@Jay SenSharma I'm also facing the same issue. Would be great to receive your help. While I see that other services (e.g. Ambari-admin) gets build fine and even see RPM been built; it fail for metrics timelineservice. ps. timelineservice jar is built, it's the rpm build phase that is failing.
... View more
09-15-2017
09:19 AM
Is it must to try for 2.5.2.0? The problem with 2.5.2.0 I faced is I couldn't even do "yum install ambari-server" (ref: thread:https://community.hortonworks.com/questions/136145/unable-to-install-ambari-server-and-agent.html). Currently I have 2.0.0 and for that ambari-server is installed but when I did try manual installation of agent "yum install ambari-agent", it always returns "No package ambari-agent available." despite trying it afer cleaning cache/metadata etc.
... View more
09-15-2017
06:28 AM
Re-checked my passwordless ssh from all hosts. There wasn't any issue from main host i.e. HW-PC-NameNode, However noticed some issues from data nodes, which I resolved now. I'm not getting "ERROR MESSAGE: tcgetattr:Invalid argument" message while doing ssh. I tried running host registration through Ambari again post above; but facing the same issue again. Please, need your help.
... View more
09-14-2017
11:49 AM
Thanks for your quick response. HW-PC-NameNode.pc.com is the one where I also installed ambari server. I did have issues initially with doing passwordless ssh from itself, but I remember I did resolve that yesterday but will do double check and if it fails again then will post. do not think it'd pop-up again though. Already added fqdn alongwith IP addresses in /etc/hosts on each of the nodes and ensured that "hostname -f" returns correct fqdn i.e. in this case it will be HW-PC-NameNoode.pc.com.
... View more
09-14-2017
08:47 AM
My OS: Centos 6.9 32 bit JDK: 1.8 I have already made cluster of 5 instanced (on virtualbox) ready including passwordless ssh, iptables turned off, ntpd started etc. Because Ambari's latest version was running into problems (another question raised on the community), I chose version 2.0.0 using which I was able to successfully install ambari-server and also done setup. Also installed correct (based on combination given in installation doc) repo files for hdp and ambari on all hosts. When I started ambari-server, access through GUI and reached to registration page (i.e. confim hosts); it failed for all hosts. Failure message (part of it) is: scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py
host=hw-pc-namenode.pc.com, exitcode=0
Command end time 2017-09-14 13:53:59
==========================
Running setup agent script...
==========================
Command start time 2017-09-14 13:53:59
/bin/sh: /usr/sbin/ambari-agent: No such file or directory
Error: No matching Packages to list
Error: No matching Packages to list
('', None)
Connection to hw-pc-namenode.pc.com closed.
SSH command execution finished
host=hw-pc-namenode.pc.com, exitcode=1
Command end time 2017-09-14 13:54:04
ERROR: Bootstrap of host hw-pc-namenode.pc.com fails because previous action finished with non-zero exit code (1)
ERROR MESSAGE: tcgetattr: Invalid argument
Connection to hw-pc-namenode.pc.com closed.
STDOUT: /bin/sh: /usr/sbin/ambari-agent: No such file or directory
Error: No matching Packages to list
Error: No matching Packages to list
('', None)
Connection to hw-pc-namenode.pc.com closed.
Since the failure is about finding ambari-agent file, I tried manually install using "yum install ambari-agent*" (Also tried yum clean all prior to this). However it was giving message ambari-agent not found. Hence used the tarball link found in doc for Ambari 2.0.0, download the tarball and glanced through it on local machine - it does not seem to have ambari-agent rpm. tried searching google, but no use. Similarly I check few of the other versions of tarballs and found only 1.x has ambari-agent rpm. I'm not much good at linux, so this left me confused. ps: I'm not behind firewall which I did check during investigation on my last question. Can you please help me with: 1. How should I overcome this error of ambari-agent (for 2.0.0) not found? 2. Normally, How is yum supposed to work n download ambari agent when it is not found in tarball as well (just for my knowledge)? Many thanks in advance of your help.
... View more
Labels:
09-13-2017
04:49 AM
Just to filter possibility of any issues with the setup of that particular instance of Centos, I tried installing on another instance as well, but it still fails. Also to nullify any connection issues, I tried curl on the GPG key from ambari.repo - it works fine and downloads the key. Surprise.. Surprise.. I just tried Ambari 2.0.0 installation and it works fine.. So it may be problem with Ambari 2.5.2.0 (In fact entire 2.x I think) repository.
... View more
09-12-2017
02:49 PM
Thanks. Done, but no success yet.
... View more
09-12-2017
02:21 PM
changed resolv.conf and tried above commands. Still no success.
... View more
09-12-2017
01:42 PM
# cat /etc/resolv.conf
; generated by /sbin/dhclient-script
search domain.name pc.com
nameserver 192.168.1.1 domain name above is dummy one and nameserver IP is actually that of my router. could that be a problem? But... I remember having worked in similar setup in past, though long back. # yum clean all
Loaded plugins: fastestmirror, security
Cleaning repos: ambari-2.5.2.0 base extras updates
Cleaning up Everything
Cleaning up list of fastest mirror # yum clean metadata
Loaded plugins: fastestmirror, security
Cleaning repos: ambari-2.5.2.0 base extras updates
0 metadata files removed
0 sqlite files removed
0 metadata files removed yum install -y ambari-server
Loaded plugins: fastestmirror, security
Setting up Install Process
Determining fastest mirrors
* base: mirrors.viethosting.com
* extras: mirror.digistar.vn
* updates: mirrors.vinahost.vn
ambari-2.5.2.0 | 2.9 kB 00:00
ambari-2.5.2.0/primary_db | 8.6 kB 00:00
base | 3.7 kB 00:00
base/primary_db | 3.7 MB 00:15
extras | 3.3 kB 00:00
extras/primary_db | 21 kB 00:00
updates | 3.4 kB 00:00
updates/primary_db | 3.0 MB 00:25
No package ambari-server available.
Error: Nothing to do
... View more
09-12-2017
01:10 PM
No. The link confirms "No Proxy server detected". Also re-checked my connection and found it's ok; but still the same result.
... View more
09-12-2017
12:50 PM
wget command works fine and downloads repomd.xml. grep for 'proxy' returns nothing. Actually I'm working from home, so usual corporate network restrictions do not apply in my case.
... View more
09-12-2017
11:58 AM
Thanks for your response Geoffrey. cat /etc/yum.repos.d/ambari.repo
#VERSION_NUMBER=2.5.2.0-298
[ambari-2.5.2.0]
name=ambari Version - ambari-2.5.2.0
baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.2.0
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.2.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
I do not have HDP repo downloaded, hence I simply get following for hdp repository: cat: /etc/yum.repos.d/hdp.repo: No such file or directory After doing yum clean all and checking yum repolist (output of it below); I'm still getting message "No package ambari-server available. # yum repolist
Loaded plugins: fastestmirror, security
Determining fastest mirrors
* base: centos-hcm.viettelidc.com.vn
* extras: mirrors.nhanhoa.com
* updates: mirrors.nhanhoa.com
ambari-2.5.2.0 | 2.9 kB 00:00
ambari-2.5.2.0/primary_db | 8.6 kB 00:00
base | 3.7 kB 00:00
base/primary_db | 3.7 MB 00:07
extras | 3.3 kB 00:00
extras/primary_db | 21 kB 00:00
updates | 3.4 kB 00:00
updates/primary_db | 3.0 MB 00:07
repo id repo name status
ambari-2.5.2.0 ambari Version - ambari-2.5.2.0 12
base CentOS-6 - Base 5,079
extras CentOS-6 - Extras 20
updates CentOS-6 - Updates 485
repolist: 5,596
# yum install -y ambari-server
Loaded plugins: fastestmirror, security
Setting up Install Process
Loading mirror speeds from cached hostfile
* base: centos-hcm.viettelidc.com.vn
* extras: mirrors.nhanhoa.com
* updates: mirrors.nhanhoa.com
No package ambari-server available.
Error: Nothing to do
... View more
09-12-2017
11:09 AM
Thanks for your quick response. * I checked "/etc/yum.repos.d/ambari.repo" and it does have enabled flag = 1. However I do not have any ambari.repo file in /tmp. * I did "yum clean all", checked ambari appears in repolist and ran "yum install ambari-server", but still the same failure.
... View more
09-12-2017
10:50 AM
1 Kudo
centos-screen1.pngcentos-screen2.png
... View more
Labels: