About nismaily

nismaily · ‎08-19-2019

This article describes the configuration of the authorizers.xml for using the Composite User Group Provider for both LDAP and File based authentication in Cloudera Manager for CDF 1.0 This article assumes TLS has already been configured for NiFi using either the NiFi CA or your own certs. Here is the Active Directory Structure that I will use for my sync: Here are the groups in the Groups OU: Here are the users in the Users OU: Here are the service users in the ServiceUsers OU: First, I will import my Active Directory root certificate into the NiFi and Java Truststores: keytool -import -file adcert.pem -alias ad -keystore /var/lib/nifi/cert/truststore.jks keytool -import -file adcert.pem -alias ad -keystore /etc/pki/java/cacerts Now login to Cloudera Manager and proceed to the NiFi Configuration. Update the configuration to use the "composite-user-group-provider" as follows: xml.authorizers.accessPolicyProvider.file-access-policy-provider.property.User Group Provider=composite-user-group-provider Next, use the "NiFi Node Advanced Configuration Snippet (Safety Valve) for staging/authorizers.xml" and add the following configurations using the XML view: <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.class</name> <value>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Manager DN</name> <value>CN=ldapbind-svc,OU=ServiceUsers,OU=CLDR,DC=nismaily,DC=com</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Manager Password</name> <value>hadoop</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Referral Strategy</name> <value>FOLLOW</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Connect Timeout</name> <value>10 secs</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Read Timeout</name> <value>10 secs</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Url</name> <value>ldaps://win-ltfjo4jgo4r.nismaily.com:636</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Sync Interval</name> <value>15 mins</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.User Search Base</name> <value>OU=Users,OU=CLDR,DC=nismaily,DC=com</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.User Object Class</name> <value>user</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.User Search Scope</name> <value>SUBTREE</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.User Identity Attribute</name> <value>sAMAccountName</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.User Group Name Attribute</name> <value>memberof</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Group Search Base</name> <value>OU=Groups,OU=CLDR,DC=nismaily,DC=com</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Group Object Class</name> <value>group</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Group Search Scope</name> <value>SUBTREE</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Group Name Attribute</name> <value>cn</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Group Member Attribute</name> <value>member</value> </property> <property> <name>xml.authorizers.userGroupProvider.composite-user-group-provider.class</name> <value>org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider</value> </property> <property> <name>xml.authorizers.userGroupProvider.composite-user-group-provider.property.Configurable User Group Provider</name> <value>file-user-group-provider</value> </property> <property> <name>xml.authorizers.userGroupProvider.composite-user-group-provider.property.User Group Provider 1</name> <value>ldap-user-group-provider</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.Authentication Strategy</name> <value>LDAPS</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Keystore</name> <value>/var/lib/nifi/cert/keystore.jks</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Keystore Password</name> <value>hadoop</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Keystore Type</name> <value>jks</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Truststore</name> <value>/var/lib/nifi/cert/truststore.jks</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Truststore Password</name> <value>hadoop</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Truststore Type</name> <value>jks</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Client Auth</name> <value>WANT</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Protocol</name> <value>TLSv1.2</value> </property> <property> <name>xml.authorizers.userGroupProvider.ldap-user-group-provider.property.TLS - Shutdown Gracefully</name> <value>false</value> </property> Restart NiFi and login to the NiFi UI. Proceed to the Users Tab: You will see the users synced from Active Directory:

andreas_maier · ‎07-14-2017

Answering my own question: It doesn't work with the latest version of cloudbreak (1.16.1). After login to the GUI I do get the error "Cannot retrieve csrf token" . But it does work with version 1.14.4 .

nismaily · ‎03-25-2017

Did you copy the ojdbc jar to the nifi lib directory and restart nifi?

rzang · ‎06-29-2017

Note that Manage Journal Node is now available through UI in Ambari 2.5.1 https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-operations/content/manage_journal_nodes.html

erkansirin78 · ‎06-20-2018

Hi guys same problem that I had. I tried many things. Finally I changed my yarn llap queue max capacity from %50 to %100 and then Hive2Interactive Server successfully started. Possible cause in my case: allocated containers exceeded llap queue max capacity.

nismaily · ‎06-01-2016

Thanks Ravi, I had to: 1) Copy spark shuffle jars to nodemanager classpaths on all nodes 2) add spark_shuffle to yarn.nodemanager.aux-services, set yarn.nodemanager.aux-services.spark_shuffle.class to org.apache.spark.network.yarn.YarnShuffleService in yarn-site.xml (via Ambari) 3) Restart all nodemanagers 4) Add the following to spark-defaults.conf spark.dynamicAllocation.enabled true spark.shuffle.service.enabled true 5) Set these parameters per job basis spark.dynamicAllocation.initialExecutors=# spark.dynamicAllocation.minExecutors=#

abajwa · ‎05-26-2016

Thanks @Vipin Rathor: while the guide uses the "manual" option of security wizard, it may be useful to know that starting Ambari 2.4, Ambari security wizard will support FreeIPA as one of the options for kerborizing (along with ActiveDir, MIT KDC). This will greatly ease the process of integrating FreeIPA with HDP

cstanca · ‎01-25-2017

@Nasheb Ismaily Yes. It is expected for FIFO policy. If you set a FIFO policy, then jobs are executed in the order you submitted them. You have the option to use FAIR policy. In that case, all jobs can be executed sharing fairly available resources and they don't have to wait one after the other. They will still start in the order you submitted, but based on what they do, they may finish in a different order. That assumes your cluster has enough resources and by design you wanted to go that way. I did not include references to various documents because they were already provided and are widely available.

nismaily · ‎05-12-2016

Thanks Ravi!

nismaily · ‎05-12-2016

Alon, what are you attempting to do? Is fs.hdfs.impl the correct property you are looking to change?

Online	Offline
Last Visited	‎10-13-2022 03:28 PM

Member Since	‎04-11-2016 05:39 PM
Last Visited	‎10-13-2022 03:28 PM
Posts	32
Kudos received	21

Cloudera Community

Re: Ingesting calculation views from Hana into Hiv...

Re: Nifi Oracle JDBC Connection Issue

Re: tez view timeout

Re: Spark - YARN Capacity Scheduler

Re: Storm Bolt throws an error when trying to read...

NiFi Composite Authorization - LDAP and File - Usi...

Re: Using a local instance of Cloudbreak with Vagr...

Re: Nifi Oracle JDBC Connection Issue

Re: Moving Journal Node Directories

Re: Hive LLAP fails to start with RUNNING_PARTIAL ...

Re: Spark - YARN Capacity Scheduler

Re: HDP - FreeIPA integration

Re: Yarn Queue Capacity Scheduling

Re: Encryption Overhead Metrics

Re: where can I find fs.hdfs.impl property