Member since
01-04-2021
29
Posts
4
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
944 | 09-17-2023 03:09 AM |
01-10-2024
08:29 PM
@MattWho @bbende Any thoughts on this?
... View more
12-26-2023
10:48 PM
Hi Everyone, I have been trying to explore the possibility of exploring apache ranger for centralized authorization in a microservice architecture. I'm think thinking of two possibilities: 1) Integrating ranger plugin with all the services and authorize the request coming to them by intercepting the request. The drawback here is that I will have to implement this common functionality in all the applications. 2) Creating a separate application for authorization, which will implement the ranger sdk to pull all the policies and configurations from Apache ranger and authorize a request using the ranger sdk, after authorization forward the request to the microservice. The drawback here is that I may not be able to use the existing plugins for Nifi, Kafka etc. when trying to use a centralized solution. What would be the best approach to implement, if someone has implemented something similar can you provide some thought on this? Or is there an even better way of doing this?
... View more
Labels:
- Labels:
-
Apache Ranger
12-01-2023
10:44 PM
I have been trying to build a custom plugin for a inhouse tool that we have built for apache ranger. I have been reading various articles and found pretty good ones including the one here. From what I understand the importance of specifying pollIntervalMs is to refresh the policies in Apache ranger admin periodically, hence without restarting Ranger admin. But when considering an example, as seen in the case of Nifi the polices are cached in a temporary directory in Nifi. Is this caching really required when developing the plugins? Is this caching done by this pollIntervalMs or is it something specific to Nifi? @MattWho @bbende Any thoughts on this?
... View more
Labels:
- Labels:
-
Apache Ranger
09-21-2023
12:35 AM
Looks like security zone is also working with nifi when configuring with ranger provided all the resource identifiers that needs to be added is also added in the security zone that we are creating. Probably this is a basic that I left out. Currently the resources identifiers in policies as shown below. But the security zone has only /flow as shown below The policies json corresponding to the above configuration is shown below. {"serviceName":"nifi","serviceId":1,"policyVersion":140,"policyUpdateTime":"20230921-12:51:41.126-+0530","policies":[],"serviceDef":{"name":"nifi","displayName":"nifi","implClass":"org.apache.ranger.services.nifi.RangerServiceNiFi","label":"NIFI","description":"NiFi","options":{"enableDenyAndExceptionsInPolicies":"false"},"configs":[{"itemId":400,"name":"nifi.url","type":"string","mandatory":true,"defaultValue":"http://localhost:8080/nifi-api/resources","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"The URL of the NiFi REST API that provides the available resources.\"}","label":"NiFi URL"},{"itemId":410,"name":"nifi.authentication","type":"enum","subType":"authType","mandatory":true,"defaultValue":"NONE","label":"Authentication Type"},{"itemId":411,"name":"nifi.ssl.use.default.context","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"If true, then Ranger\u0027s keystore and truststore will be used to communicate with NiFi. If false, the keystore and truststore properties must be provided.\"}","label":"Use Ranger\u0027s Default SSL Context"},{"itemId":500,"name":"nifi.ssl.keystore","type":"string","mandatory":false,"label":"Keystore"},{"itemId":510,"name":"nifi.ssl.keystoreType","type":"string","mandatory":false,"label":"Keystore Type"},{"itemId":520,"name":"nifi.ssl.keystorePassword","type":"password","mandatory":false,"label":"Keystore Password"},{"itemId":530,"name":"nifi.ssl.truststore","type":"string","mandatory":false,"label":"Truststore"},{"itemId":540,"name":"nifi.ssl.truststoreType","type":"string","mandatory":false,"label":"Truststore Type"},{"itemId":550,"name":"nifi.ssl.truststorePassword","type":"password","mandatory":false,"label":"Truststore Password"},{"itemId":560,"name":"ranger.plugin.audit.filters","type":"string","mandatory":false,"defaultValue":"[]","label":"Ranger Default Audit Filters"}],"resources":[{"itemId":100,"name":"nifi-resource","type":"string","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":false,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"true"},"label":"NiFi Resource Identifier","description":"NiFi Resource","accessTypeRestrictions":[],"isValidLeaf":true}],"accessTypes":[{"itemId":100,"name":"READ","label":"Read","impliedGrants":[]},{"itemId":200,"name":"WRITE","label":"Write","impliedGrants":[]}],"policyConditions":[{"itemId":1,"name":"_expression","evaluator":"org.apache.ranger.plugin.conditionevaluator.RangerScriptConditionEvaluator","evaluatorOptions":{"ui.isMultiline":"true"},"uiHint":"{ \"isMultiline\":true }","label":"Enter boolean expression","description":"Boolean expression"}],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authType","elements":[{"itemId":1,"name":"NONE","label":"None"},{"itemId":2,"name":"SSL","label":"SSL"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]},"id":10,"guid":"a23a32e1-4b74-4cbf-8b60-3e161d748954","isEnabled":true,"createTime":"20230912-17:43:16.455-+0530","updateTime":"20230912-17:43:16.798-+0530","version":1},"auditMode":"audit-default","securityZones":{"nifi":{"zoneName":"nifi","resources":[{"nifi-resource":["/flow"]}],"policies":[{"service":"nifi","name":"nifi-zone-policy","policyType":0,"policyPriority":0,"isAuditEnabled":true,"resources":{"nifi-resource":{"values":["/proxy"],"isExcludes":false,"isRecursive":false}},"additionalResources":[{"nifi-resource":{"values":["/flow"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/data/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}}],"policyItems":[{"accesses":[{"type":"READ","isAllowed":true},{"type":"WRITE","isAllowed":true}],"users":["vishnu"],"groups":[],"roles":["ranger_users"],"conditions":[],"delegateAdmin":true}],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[],"rowFilterPolicyItems":[],"serviceType":"nifi","options":{},"validitySchedules":[],"policyLabels":[],"zoneName":"nifi","isDenyAllElse":false,"id":4,"guid":"b4577f50-f5fc-40ae-b745-7ec9380e2705","isEnabled":true,"version":13}],"containsAssociatedTagService":false}}} Since only flow was in the security zone the other configurations was not working. If I add other two resource specifiers in the nifi security zone, then then things are working properly. Also not sure if there is some other thing that you mentioned as security zone.
... View more
09-20-2023
01:01 PM
@MattWho Thank you once again for making my life easier . Looks like the problem is caused by setting security zones in ranger. Once I created polices outside the security zone things are working fine. Is there any reason nifi doesn't support security zones? And do you have any idea why only /flow was working inside security zone?
... View more
09-20-2023
12:21 PM
@MattWho I had already configured the process groups endpoints to the resource identifiers, all the four shown in nifi ranger plugin. The identifier processor group shown in nifi is below. Some of the resource identifiers I configured on nifi ranger service is shown below. The corresponding allow conditions are shown below. This was the configurations I was testing in ranger service definition which I believe is what is specified in the answer. But as I said the /flow resource specifier works perfectly fine. Is there any other point where there is a chance of this going wrong or is there something wrong in whatever I did?
... View more
09-20-2023
11:19 AM
Currently I have configured apache NiFi with Ranger as per discussed in this thread. Ranger NiFi plugin is able to pull resources from ranger and auto suggest in ranger nifi service definition UI. But I don't think the authorizations works properly.
Only the /flow resource specifier has an impact. When I remove that specifier user is not able to view the screen and when I add it back the user is logging back into the Nifi UI. Rest of the resources including processors , ports etc. seems to be unclickable or accessible no matter any of the resource specifier I provide in ranger. I even tried the wild card * but still doesn't seems to work. The UI looks as below.
The Ranger Authorizers file is shown below
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1"></property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity"></property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
<property name="Node Group"></property>
</accessPolicyProvider>
<authorizer>
<identifier>ranger-provider</identifier>
<class>org.apache.nifi.ranger.authorization.RangerNiFiAuthorizer</class>
<property name="Ranger Audit Config Path">./conf/ranger-nifi-audit.xml</property>
<property name="Ranger Security Config Path">./conf/ranger-nifi-security.xml</property>
<property name="Ranger Service Type">nifi</property>
<property name="Ranger Application Id">nifi</property>
<property name="Ranger Admin Identity">CN=localhost, OU=NIFI</property>
<property name="Ranger Kerberos Enabled">false</property>
</authorizer>
The initial admin is specified for resource access through user present in the certificate.
@MattWho any thoughts on this?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Oozie
-
Apache Ranger
09-19-2023
09:40 PM
Thank you @MattWho for your valuable insights. Integrations are working fine now. But before we end this thread I have a few more questions that you can possibly answer. 1) Currently I have configured an SSL user and users from LDAP. When logging in browser during loading nifi the first option that comes in is sign in with the SSL user. Is there a way to disable it? 2) Currently I have added the SSL user as initial admin identity, then removed excessive permissions from authorizations.xml. Are there any best practice for achieving both? Also I don't think apart from /flow other resource access are working properly. I added the resource paths for creating processors but doesn't seem to work.
... View more
09-18-2023
10:32 PM
@MattWho the 401 issue was a silly issue with specifying truststore instead of keystore and viceversa. But now it is throwing SSL Handshake exeception as below. Since both nifi and ranger is running on my local machine I configured the same truststore and keystore for both. Update: Not sure what is wrong with the SSL certs I created using openSSL, but when I create certifcates with nifi tool kit the exception has gone. Now its throwing 403.
... View more
09-18-2023
12:31 AM
Looks like my issue is with specifying policy.download.auth.users as nifi. I was integrating pluggin following this article. I skipped the step 7 Give user and group ownership with nifi process user and set permission 400 to files ranger-nifi-audit.xml and ranger-nifi-security.xml I tried adding a new user nifi and group nifi in my ubuntu and permission 400 and ownership to nifi user for files ranger-nifi-audit.xml and ranger-nifi-security.xml. But still its throwing 401. Update: I modified the bootstrap.conf file in Nifi to update run.as property to nifi and other users as well. But still there isn't any effect. @MattWho @bbende any thoughts on this?
... View more