Member since
01-04-2021
26
Posts
4
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
81 | 09-17-2023 03:09 AM |
09-21-2023
12:35 AM
Looks like security zone is also working with nifi when configuring with ranger provided all the resource identifiers that needs to be added is also added in the security zone that we are creating. Probably this is a basic that I left out. Currently the resources identifiers in policies as shown below. But the security zone has only /flow as shown below The policies json corresponding to the above configuration is shown below. {"serviceName":"nifi","serviceId":1,"policyVersion":140,"policyUpdateTime":"20230921-12:51:41.126-+0530","policies":[],"serviceDef":{"name":"nifi","displayName":"nifi","implClass":"org.apache.ranger.services.nifi.RangerServiceNiFi","label":"NIFI","description":"NiFi","options":{"enableDenyAndExceptionsInPolicies":"false"},"configs":[{"itemId":400,"name":"nifi.url","type":"string","mandatory":true,"defaultValue":"http://localhost:8080/nifi-api/resources","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"The URL of the NiFi REST API that provides the available resources.\"}","label":"NiFi URL"},{"itemId":410,"name":"nifi.authentication","type":"enum","subType":"authType","mandatory":true,"defaultValue":"NONE","label":"Authentication Type"},{"itemId":411,"name":"nifi.ssl.use.default.context","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"If true, then Ranger\u0027s keystore and truststore will be used to communicate with NiFi. If false, the keystore and truststore properties must be provided.\"}","label":"Use Ranger\u0027s Default SSL Context"},{"itemId":500,"name":"nifi.ssl.keystore","type":"string","mandatory":false,"label":"Keystore"},{"itemId":510,"name":"nifi.ssl.keystoreType","type":"string","mandatory":false,"label":"Keystore Type"},{"itemId":520,"name":"nifi.ssl.keystorePassword","type":"password","mandatory":false,"label":"Keystore Password"},{"itemId":530,"name":"nifi.ssl.truststore","type":"string","mandatory":false,"label":"Truststore"},{"itemId":540,"name":"nifi.ssl.truststoreType","type":"string","mandatory":false,"label":"Truststore Type"},{"itemId":550,"name":"nifi.ssl.truststorePassword","type":"password","mandatory":false,"label":"Truststore Password"},{"itemId":560,"name":"ranger.plugin.audit.filters","type":"string","mandatory":false,"defaultValue":"[]","label":"Ranger Default Audit Filters"}],"resources":[{"itemId":100,"name":"nifi-resource","type":"string","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":false,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"true"},"label":"NiFi Resource Identifier","description":"NiFi Resource","accessTypeRestrictions":[],"isValidLeaf":true}],"accessTypes":[{"itemId":100,"name":"READ","label":"Read","impliedGrants":[]},{"itemId":200,"name":"WRITE","label":"Write","impliedGrants":[]}],"policyConditions":[{"itemId":1,"name":"_expression","evaluator":"org.apache.ranger.plugin.conditionevaluator.RangerScriptConditionEvaluator","evaluatorOptions":{"ui.isMultiline":"true"},"uiHint":"{ \"isMultiline\":true }","label":"Enter boolean expression","description":"Boolean expression"}],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authType","elements":[{"itemId":1,"name":"NONE","label":"None"},{"itemId":2,"name":"SSL","label":"SSL"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]},"id":10,"guid":"a23a32e1-4b74-4cbf-8b60-3e161d748954","isEnabled":true,"createTime":"20230912-17:43:16.455-+0530","updateTime":"20230912-17:43:16.798-+0530","version":1},"auditMode":"audit-default","securityZones":{"nifi":{"zoneName":"nifi","resources":[{"nifi-resource":["/flow"]}],"policies":[{"service":"nifi","name":"nifi-zone-policy","policyType":0,"policyPriority":0,"isAuditEnabled":true,"resources":{"nifi-resource":{"values":["/proxy"],"isExcludes":false,"isRecursive":false}},"additionalResources":[{"nifi-resource":{"values":["/flow"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/data/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}}],"policyItems":[{"accesses":[{"type":"READ","isAllowed":true},{"type":"WRITE","isAllowed":true}],"users":["vishnu"],"groups":[],"roles":["ranger_users"],"conditions":[],"delegateAdmin":true}],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[],"rowFilterPolicyItems":[],"serviceType":"nifi","options":{},"validitySchedules":[],"policyLabels":[],"zoneName":"nifi","isDenyAllElse":false,"id":4,"guid":"b4577f50-f5fc-40ae-b745-7ec9380e2705","isEnabled":true,"version":13}],"containsAssociatedTagService":false}}} Since only flow was in the security zone the other configurations was not working. If I add other two resource specifiers in the nifi security zone, then then things are working properly. Also not sure if there is some other thing that you mentioned as security zone.
... View more
09-20-2023
01:01 PM
@MattWho Thank you once again for making my life easier . Looks like the problem is caused by setting security zones in ranger. Once I created polices outside the security zone things are working fine. Is there any reason nifi doesn't support security zones? And do you have any idea why only /flow was working inside security zone?
... View more
09-20-2023
12:21 PM
@MattWho I had already configured the process groups endpoints to the resource identifiers, all the four shown in nifi ranger plugin. The identifier processor group shown in nifi is below. Some of the resource identifiers I configured on nifi ranger service is shown below. The corresponding allow conditions are shown below. This was the configurations I was testing in ranger service definition which I believe is what is specified in the answer. But as I said the /flow resource specifier works perfectly fine. Is there any other point where there is a chance of this going wrong or is there something wrong in whatever I did?
... View more
09-20-2023
11:19 AM
Currently I have configured apache NiFi with Ranger as per discussed in this thread. Ranger NiFi plugin is able to pull resources from ranger and auto suggest in ranger nifi service definition UI. But I don't think the authorizations works properly.
Only the /flow resource specifier has an impact. When I remove that specifier user is not able to view the screen and when I add it back the user is logging back into the Nifi UI. Rest of the resources including processors , ports etc. seems to be unclickable or accessible no matter any of the resource specifier I provide in ranger. I even tried the wild card * but still doesn't seems to work. The UI looks as below.
The Ranger Authorizers file is shown below
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1"></property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity"></property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
<property name="Node Group"></property>
</accessPolicyProvider>
<authorizer>
<identifier>ranger-provider</identifier>
<class>org.apache.nifi.ranger.authorization.RangerNiFiAuthorizer</class>
<property name="Ranger Audit Config Path">./conf/ranger-nifi-audit.xml</property>
<property name="Ranger Security Config Path">./conf/ranger-nifi-security.xml</property>
<property name="Ranger Service Type">nifi</property>
<property name="Ranger Application Id">nifi</property>
<property name="Ranger Admin Identity">CN=localhost, OU=NIFI</property>
<property name="Ranger Kerberos Enabled">false</property>
</authorizer>
The initial admin is specified for resource access through user present in the certificate.
@MattWho any thoughts on this?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Oozie
-
Apache Ranger
09-19-2023
09:40 PM
Thank you @MattWho for your valuable insights. Integrations are working fine now. But before we end this thread I have a few more questions that you can possibly answer. 1) Currently I have configured an SSL user and users from LDAP. When logging in browser during loading nifi the first option that comes in is sign in with the SSL user. Is there a way to disable it? 2) Currently I have added the SSL user as initial admin identity, then removed excessive permissions from authorizations.xml. Are there any best practice for achieving both? Also I don't think apart from /flow other resource access are working properly. I added the resource paths for creating processors but doesn't seem to work.
... View more
09-18-2023
10:32 PM
@MattWho the 401 issue was a silly issue with specifying truststore instead of keystore and viceversa. But now it is throwing SSL Handshake exeception as below. Since both nifi and ranger is running on my local machine I configured the same truststore and keystore for both. Update: Not sure what is wrong with the SSL certs I created using openSSL, but when I create certifcates with nifi tool kit the exception has gone. Now its throwing 403.
... View more
09-18-2023
12:31 AM
Looks like my issue is with specifying policy.download.auth.users as nifi. I was integrating pluggin following this article. I skipped the step 7 Give user and group ownership with nifi process user and set permission 400 to files ranger-nifi-audit.xml and ranger-nifi-security.xml I tried adding a new user nifi and group nifi in my ubuntu and permission 400 and ownership to nifi user for files ranger-nifi-audit.xml and ranger-nifi-security.xml. But still its throwing 401. Update: I modified the bootstrap.conf file in Nifi to update run.as property to nifi and other users as well. But still there isn't any effect. @MattWho @bbende any thoughts on this?
... View more
09-17-2023
09:07 AM
I have been trying to integrate Nifi with Apache Ranger. When I manually configure policies in Ranger for Nifi the policies are fetched by Nifi and authorization works fine. But when I try to define the service definition and test the connection its is giving the following error. The configuration in service definition is shown below. The authentication in nifi is setup by following this article. Note: Only Nifi is setup in SSL mode. Apache Ranger doesn't have SSL mode. What could be the possible reason this is happening?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Ranger
09-17-2023
03:09 AM
1 Kudo
After some more time of debugging, looks like all the configurations where correct. The password for the configured user was wrong in LDAP causing the issue.
... View more
09-16-2023
02:34 PM
Here is my login-identity-providers.xml <provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">cn=admin,dc=example,dc=com</property>
<property name="Manager Password">secret</property>
<property name="TLS - Keystore"></property>
<property name="TLS - Keystore Password"></property>
<property name="TLS - Keystore Type"></property>
<property name="TLS - Truststore"></property>
<property name="TLS - Truststore Password"></property>
<property name="TLS - Truststore Type"></property>
<property name="TLS - Client Auth"></property>
<property name="TLS - Protocol"></property>
<property name="TLS - Shutdown Gracefully"></property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldap://localhost:389</property>
<property name="User Search Base">cn=vishnu,cn=admin,dc=example,dc=com</property>
<property name="User Search Filter">(objectClass=*)</property>
<property name="Identity Strategy">USE_USERNAME</property>
<property name="Authentication Expiration">12 hours</property>
<property name="User Object Class">person</property>
<property name="User Search Scope">ONE_LEVEL</property>
<property name="User Identity Attribute">cn</property>
</provider> The authorizers is shown below. <userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">cn=vishnu,cn=admin,dc=example,dc=com</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">cn=vishnu,cn=admin,dc=example,dc=com</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
<property name="Node Group"></property>
</accessPolicyProvider>
<authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="User Group Provider">ldap-user-group-provider</property>
<property name="Access Policy Provider">file-access-policy-provider</property>
<property name="Initial Admin Identity">cn=vishnu,cn=admin,dc=example,dc=com</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
</authorizer> The following values of properties are updated nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml Below is the view of LDAP from Apache Directory studio. Currently there is only one user in that search base. Can someone help identify why the authentication is failing? I referred other articles within Cloudera community and outside but none seem to be working.
... View more
Labels:
- Labels:
-
Apache NiFi