Member since
12-12-2015
27
Posts
7
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1796 | 12-12-2016 05:53 PM | |
1099 | 10-21-2016 08:53 AM |
02-11-2019
08:51 AM
Hello, To fix this issue, I used to stop/start the Flume process on a daily basis. Recently we have migrated from Flume to NiFi; much more stable. Rgds Laurent
... View more
08-01-2018
02:07 PM
Hello, I am trying to retrieve a specific field from my LDAP directory (cn), but it doesn't seem to be taken into account by the Ranger usersync process when I specify the following value "cn" to the variable "Username Attribute" Though this value seems allowed (possible values : uid or cn) it doesn't work on my HDP 2.6.2.0; I still see the field "uid" feeding the Ranger database. Is it a bug or did I miss something ? Thanks in advance for your insights. LC
... View more
Labels:
- Labels:
-
Apache Ranger
08-02-2017
02:39 PM
Hello @rbiswas, Sorry for the delay in getting back to you (I was on holidays). Thanks for your answers. Yes we can close the thread. regards Laurent
... View more
07-20-2017
08:37 AM
Hello @rbiswas, Sorry, I'm a bit confused by your last statement. Could you please confirm that if I define a replication factor of 4, and 4 racks, I will get the following distribution of replicas ? (see diagram below) regards Laurent
... View more
07-17-2017
08:19 AM
thanks @rbiswas for your answer. My concern is regarding the speed of the replication if, let's say one rack is unavailable during 24 / 48hours for maintenance reasons, and in the meantime HDFS is trying to replicate all then data on the remaining rack, thus might saturate the disk space on this rack ! I can't find any documentation mentionning this " HDFS rebalance speed" . Also it looks to me that, if the number of replica factor is equal to the number of racks, there is no guarantee that there will be a replica spread in each rack. Do you confirm it ? Thanks in advance. rgds Laurent
... View more
07-13-2017
09:06 AM
Hello, I am in the process of improving the resilience of our hadoop clusters. We are using a twin-datacenter architecture; the hadoop cluster nodes are located in two different buildings separated by 10 km with Namenode HA activated. We are using a replica factor of 4 + 2 rack awareness (on rack per site). The replica factor of 4 is probably a bit "luxury", but it might protect against the lost of an entire rack (lost of a site) + the lost of some nodes on the remaining site. In case of losing en entire rack, I am wondering if HDFS will try to replicate the data on the remaining rack, thus we will get 4 replica on the same rack and overconsume space on the remaining rack ?...or will it "disable" the replica that is supposed to be located on the failed rack ? Does it make sense to create 4 racks (one for each replica) in order to ensure that the data will be replicated on the both sites in a balanced way (2x2) ? Many thanks in advance for your feedback. Regards Laurent
... View more
Labels:
- Labels:
-
Apache Hadoop
06-16-2017
12:06 PM
Thanks @Vipin Rathor for your elaborated answer. I understand the advantages of using Kerberos, however what I found tedious is that I need to recreate user accounts (Principals) within the Kerberos Database as well as managing new passwords policies. as I said inmy previous message, ideally I would like to configure the cluster to authenticate users against LDAP and retrieve automatically a Kerberos ticket, but I don't know if it's feasible. Regards Laurent
... View more
06-14-2017
11:41 AM
Hello @Vipin Rathor, I don't find any clear documentation about the entire setup of a kerberized cluster synchronized with LDAP (not AD) in order to retrieve kerberos token, and authorize the access via Ranger and Knox. The first stage so far is to secure WebHDFS. I don't want to generate keytabs for our hundred of users; I would like to get authenticated by LDAP, and then retrieve a Kerberos token automatically.. Any clue how to do this ? regards Lau
... View more
06-13-2017
05:18 PM
Hi, I'm starting to get really lost in setting a secured cluster; I found it really complicated to properly configure Ranger and Knox on a kerberized cluster. Is there someone who would be keen in helping me ? I would be very grateful. Thanks in advance. Rgds Laurent
... View more
Labels:
- Labels:
-
Apache Knox
-
Apache Ranger
05-04-2017
01:04 PM
Hello, I'm wondering weither it's possible to simplify the architecture by allowing Knox to retrieve a generic Kerberos token as soon as we get authenticated on the Knox gateway with our LDAP user account ? any clue ? thanks in advance. rgds Laurent
... View more