About laule75

laule75 · ‎02-11-2019

Hello, To fix this issue, I used to stop/start the Flume process on a daily basis. Recently we have migrated from Flume to NiFi; much more stable. Rgds Laurent

laule75 · ‎08-01-2018

Hello, I am trying to retrieve a specific field from my LDAP directory (cn), but it doesn't seem to be taken into account by the Ranger usersync process when I specify the following value "cn" to the variable "Username Attribute" Though this value seems allowed (possible values : uid or cn) it doesn't work on my HDP 2.6.2.0; I still see the field "uid" feeding the Ranger database. Is it a bug or did I miss something ? Thanks in advance for your insights. LC

laule75 · ‎08-02-2017

Hello @rbiswas, Sorry for the delay in getting back to you (I was on holidays). Thanks for your answers. Yes we can close the thread. regards Laurent

laule75 · ‎07-20-2017

Hello @rbiswas, Sorry, I'm a bit confused by your last statement. Could you please confirm that if I define a replication factor of 4, and 4 racks, I will get the following distribution of replicas ? (see diagram below) regards Laurent

laule75 · ‎07-17-2017

thanks @rbiswas for your answer. My concern is regarding the speed of the replication if, let's say one rack is unavailable during 24 / 48hours for maintenance reasons, and in the meantime HDFS is trying to replicate all then data on the remaining rack, thus might saturate the disk space on this rack ! I can't find any documentation mentionning this " HDFS rebalance speed" . Also it looks to me that, if the number of replica factor is equal to the number of racks, there is no guarantee that there will be a replica spread in each rack. Do you confirm it ? Thanks in advance. rgds Laurent

laule75 · ‎07-13-2017

Hello, I am in the process of improving the resilience of our hadoop clusters. We are using a twin-datacenter architecture; the hadoop cluster nodes are located in two different buildings separated by 10 km with Namenode HA activated. We are using a replica factor of 4 + 2 rack awareness (on rack per site). The replica factor of 4 is probably a bit "luxury", but it might protect against the lost of an entire rack (lost of a site) + the lost of some nodes on the remaining site. In case of losing en entire rack, I am wondering if HDFS will try to replicate the data on the remaining rack, thus we will get 4 replica on the same rack and overconsume space on the remaining rack ?...or will it "disable" the replica that is supposed to be located on the failed rack ? Does it make sense to create 4 racks (one for each replica) in order to ensure that the data will be replicated on the both sites in a balanced way (2x2) ? Many thanks in advance for your feedback. Regards Laurent

laule75 · ‎06-16-2017

Thanks @Vipin Rathor for your elaborated answer. I understand the advantages of using Kerberos, however what I found tedious is that I need to recreate user accounts (Principals) within the Kerberos Database as well as managing new passwords policies. as I said inmy previous message, ideally I would like to configure the cluster to authenticate users against LDAP and retrieve automatically a Kerberos ticket, but I don't know if it's feasible. Regards Laurent

laule75 · ‎06-14-2017

Hello @Vipin Rathor, I don't find any clear documentation about the entire setup of a kerberized cluster synchronized with LDAP (not AD) in order to retrieve kerberos token, and authorize the access via Ranger and Knox. The first stage so far is to secure WebHDFS. I don't want to generate keytabs for our hundred of users; I would like to get authenticated by LDAP, and then retrieve a Kerberos token automatically.. Any clue how to do this ? regards Lau

laule75 · ‎06-13-2017

Hi, I'm starting to get really lost in setting a secured cluster; I found it really complicated to properly configure Ranger and Knox on a kerberized cluster. Is there someone who would be keen in helping me ? I would be very grateful. Thanks in advance. Rgds Laurent

laule75 · ‎05-04-2017

Hello, I'm wondering weither it's possible to simplify the architecture by allowing Knox to retrieve a generic Kerberos token as soon as we get authenticated on the Knox gateway with our LDAP user account ? any clue ? thanks in advance. rgds Laurent

Online	Offline
Last Visited	‎03-12-2019 03:47 PM

Date Registered	‎12-12-2015 02:26 PM
Last Visited	‎03-12-2019 03:47 PM
Total Messages Posted	27
Total Kudos Received	7

Re: Hosts not listed anymore after Ambari upgrade ...

Re: Flume - Polling of data continuously

Re: Flume - Polling of data continuously

Ranger USersync - Username Attribute - cn

Re: HDFS resiliency - DR - rack aware

Re: HDFS resiliency - DR - rack aware

Re: HDFS resiliency - DR - rack aware

HDFS resiliency - DR - rack aware

Re: Assistance in setting up Kerberos - Ranger - K...

Re: Assistance in setting up Kerberos - Ranger - K...

Assistance in setting up Kerberos - Ranger - Knox

Re: Secured architecture design