Member since
01-10-2020
52
Posts
1
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1488 | 08-25-2020 09:30 AM | |
2333 | 08-21-2020 08:31 PM | |
1789 | 05-28-2020 03:53 PM |
11-23-2020
05:56 AM
@venkii User/client authentication via a user certificate and Authentication via login provider are handled completely differnetly Looking at your login-identity-providers xml, I see you have configured the following: <property name="Authentication Strategy">START_TLS</property> However, you have not configured any of the TLS properties in the provider. Are you sure "START_TLS" is what you want to be using here? Your ldap URL looks to be using just ldap:// and the default unsecured port. If that is the case the "Authentication Strategy" should be set to "SIMPLE" instead of "START_TLS". The exception points to at an SSL handshake exception. It implies that a server certificate returned by the ldap server did not match (IN either DN or SAN entry) the hostname used in your ldap URL configured in the login provider. So in this case you either need to switch to "SIMPLE" (if possible) or validate the server certificate being returned by your ldap server and setup needed TLS properties in your provider. Hope this helps, Matt
... View more
09-15-2020
08:22 PM
@TimothySpann thanks for your response. - i assume ConsumeKafkaRecord_2_0 1.9 will consume messages from the Kafka partitions based on "max poll records".in my case I have set it to 100000. please let me know if my assumption is wrong. could you also confirm if there are any limitations to pull bulk messages like at least 10K messages in a single poll? with 1.9 nifi message size is 2 MB, Avro format and 3 topics have 2 million messages each
... View more
08-25-2020
09:30 AM
@BGabor thanks for your response I'm trying to understand the difference between two blocks given in the document #BLOCK1 activeDirectoryRealm = org.apache.zeppelin.realm.ActiveDirectoryGroupRealm
activeDirectoryRealm.systemUsername = userNameA
activeDirectoryRealm.systemPassword = passwordA
activeDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COM
activeDirectoryRealm.url = ldap://ldap.test.com:389
activeDirectoryRealm.groupRolesMap = "CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"group1"
activeDirectoryRealm.authorizationCachingEnabled = false
activeDirectoryRealm.principalSuffix = @corp.company.net
##BLOCK2
ldapRealm = org.apache.zeppelin.realm.LdapGroupRealm
# search base for ldap groups (only relevant for LdapGroupRealm):
ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COM
ldapRealm.contextFactory.url = ldap://ldap.test.com:389
ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COM
ldapRealm.contextFactory.authenticationMechanism = simple also define roles/groups that you want to have in the system, like below; [roles]
admin = *
hr = *
finance = *
group1 = * . could you help me understand? i want to extract only one group and users from AD, which block should I use. thanks
... View more
08-21-2020
08:31 PM
1 Kudo
figured out the issue, related to jira https://issues.apache.org/jira/browse/NIFI-6999 flow.xml.gz is 963 mb, where toolkit mem isn't enough to restart from ambari GUI. hence started through CLI restarted the nifi from CLI rather than AMBARI , came up flow.xml.gz file size is 963MB instance : 120GB memory NIFI allocated mem: 45GB toolkit 15GB mem error: from ambari agent logs ERROR:020/08/21 19:01:30 ERROR [main] org.apache.nifi.toolkit.encryptconfig.EncryptConfigMain: java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.lang.StringCoding.encode(StringCoding.java:350) at java.lang.String.getBytes(String.java:941) at org.apache.commons.io.IOUtils.write(IOUtils.java:2025) at org.apache.commons.io.IOUtils$write$0.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:141) at org.apache.nifi.properties.ConfigEncryptionTool$_writeFlowXmlToFile_closure6$_closure30.doCall(ConfigEncryptionTool.groovy:870) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.Closure.call(Closure.java:414) at groovy.lang.Closure.call(Closure.java:430) at org.codehaus.groovy.runtime.IOGroovyMethods.withCloseable(IOGroovyMethods.java:1622) at org.codehaus.groovy.runtime.NioGroovyMethods.withCloseable(NioGroovyMethods.java:1759) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54) at org.codehaus.groovy.runtime.metaclass.NewInstanceMetaMethod.invoke(NewInstanceMetaMethod.java:56) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at org.apache.nifi.properties.ConfigEncryptionTool$_writeFlowXmlToFile_closure6.doCall(ConfigEncryptionTool.groovy:869) Requested array size exceeds VM limit
... View more
07-20-2020
03:04 PM
this is multi-az AWS cluster 3 nodes on zone 1 and 3 on zone 2
... View more
07-10-2020
12:49 AM
Thanks on hdf 3.4.1. cleared the users from the auth.xml and users.xml. back to normal operations. Appreciate your solution.
... View more
05-28-2020
03:53 PM
I am able to resolve the issue. Update nifi authorizers to use ranger ldap rather than file based.
... View more
03-30-2020
10:10 PM
Yes @MattWho, you are awesome, adding the node resolved the issue
... View more
03-27-2020
01:18 PM
exactly, for some reason though my nifi is 2 nodes secured cluster when I logged in it shows 4 nodes . two with secured and two with unsecured ports. stopped and followed the shared process. It came up clean.
... View more
01-17-2020
01:51 PM
@venkii You need to generate a certificate for each of your NiFi nodes in your NiFi cluster. While you can create just one certificate (single DN) with multiple Subject Alternative Names (SAN) entries for each host, this is not security best practices. There is no point in setting multiple identities for the same DN string. For each "Initial User Identity <num>", the configured string is added as a user in the users.xml file for each "Node Identity <num>", the configured string has authorizations needed for NiFi nodes associated to that user's uuid in the authorizations.xml file. In order to access the NiFi UI after securing NiFi, your user/client will need to authenticate with your NiFi. By default the only configured user authentication method setup is via TLS certificates which requires you to to create a certificate for each user accessing NiFi. This certificate would be loaded in to your browser and not NiFi itself (the public certificate for your user's private certificate would need to be added to the NiFi truststore in order for NiFi to trust the cert). NiFi can also be configured to support kerberos, ldap, openId connect, etc alternative methods for user authentication. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Once a user is successfully authenticated, that user must be authorized before they can even see the NiFi UI. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization I understand you have already started down this path, but I recommend you carefully read above document links for more details.
... View more