Looks like security zone is also working with nifi when configuring with ranger provided all the resource identifiers that needs to be added is also added in the security zone that we are creating. Probably this is a basic that I left out.  Currently the resources identifiers in policies as shown below.   But the security zone has only /flow as shown below     The policies json corresponding to the above configuration is shown below. {"serviceName":"nifi","serviceId":1,"policyVersion":140,"policyUpdateTime":"20230921-12:51:41.126-+0530","policies":[],"serviceDef":{"name":"nifi","displayName":"nifi","implClass":"org.apache.ranger.services.nifi.RangerServiceNiFi","label":"NIFI","description":"NiFi","options":{"enableDenyAndExceptionsInPolicies":"false"},"configs":[{"itemId":400,"name":"nifi.url","type":"string","mandatory":true,"defaultValue":"http://localhost:8080/nifi-api/resources","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"The URL of the NiFi REST API that provides the available resources.\"}","label":"NiFi URL"},{"itemId":410,"name":"nifi.authentication","type":"enum","subType":"authType","mandatory":true,"defaultValue":"NONE","label":"Authentication Type"},{"itemId":411,"name":"nifi.ssl.use.default.context","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"If true, then Ranger\u0027s keystore and truststore will be used to communicate with NiFi. If false, the keystore and truststore properties must be provided.\"}","label":"Use Ranger\u0027s Default SSL Context"},{"itemId":500,"name":"nifi.ssl.keystore","type":"string","mandatory":false,"label":"Keystore"},{"itemId":510,"name":"nifi.ssl.keystoreType","type":"string","mandatory":false,"label":"Keystore Type"},{"itemId":520,"name":"nifi.ssl.keystorePassword","type":"password","mandatory":false,"label":"Keystore Password"},{"itemId":530,"name":"nifi.ssl.truststore","type":"string","mandatory":false,"label":"Truststore"},{"itemId":540,"name":"nifi.ssl.truststoreType","type":"string","mandatory":false,"label":"Truststore Type"},{"itemId":550,"name":"nifi.ssl.truststorePassword","type":"password","mandatory":false,"label":"Truststore Password"},{"itemId":560,"name":"ranger.plugin.audit.filters","type":"string","mandatory":false,"defaultValue":"[]","label":"Ranger Default Audit Filters"}],"resources":[{"itemId":100,"name":"nifi-resource","type":"string","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":false,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"true"},"label":"NiFi Resource Identifier","description":"NiFi Resource","accessTypeRestrictions":[],"isValidLeaf":true}],"accessTypes":[{"itemId":100,"name":"READ","label":"Read","impliedGrants":[]},{"itemId":200,"name":"WRITE","label":"Write","impliedGrants":[]}],"policyConditions":[{"itemId":1,"name":"_expression","evaluator":"org.apache.ranger.plugin.conditionevaluator.RangerScriptConditionEvaluator","evaluatorOptions":{"ui.isMultiline":"true"},"uiHint":"{ \"isMultiline\":true }","label":"Enter boolean expression","description":"Boolean expression"}],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authType","elements":[{"itemId":1,"name":"NONE","label":"None"},{"itemId":2,"name":"SSL","label":"SSL"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]},"id":10,"guid":"a23a32e1-4b74-4cbf-8b60-3e161d748954","isEnabled":true,"createTime":"20230912-17:43:16.455-+0530","updateTime":"20230912-17:43:16.798-+0530","version":1},"auditMode":"audit-default","securityZones":{"nifi":{"zoneName":"nifi","resources":[{"nifi-resource":["/flow"]}],"policies":[{"service":"nifi","name":"nifi-zone-policy","policyType":0,"policyPriority":0,"isAuditEnabled":true,"resources":{"nifi-resource":{"values":["/proxy"],"isExcludes":false,"isRecursive":false}},"additionalResources":[{"nifi-resource":{"values":["/flow"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/data/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}}],"policyItems":[{"accesses":[{"type":"READ","isAllowed":true},{"type":"WRITE","isAllowed":true}],"users":["vishnu"],"groups":[],"roles":["ranger_users"],"conditions":[],"delegateAdmin":true}],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[],"rowFilterPolicyItems":[],"serviceType":"nifi","options":{},"validitySchedules":[],"policyLabels":[],"zoneName":"nifi","isDenyAllElse":false,"id":4,"guid":"b4577f50-f5fc-40ae-b745-7ec9380e2705","isEnabled":true,"version":13}],"containsAssociatedTagService":false}}} Since only flow was in the security zone the other configurations was not working. If I add other two resource specifiers in the nifi security zone, then then things are working properly. Also not sure if there is some other thing that you mentioned as security zone. ... View more

@pacman  1. There is noway to disable TLS. If you remove your TLS certificate from your browser or use and incognito window your client certificate will not be presented in the TLS exchange. NiFi requires TLS certificate authentication for NiFi to NiFi authenticated and authorized connections (for example in a multi-node NiFi cluster or utilizing NiFi Site-to-Site between different NiFi deployments).  When NiFi is secured an NO other methods of authentication are configured, NiFi will "REQUIRE" a MutualTLS exchange.  Once at least one additional method of authentication is  configured, NiFi will "WANT" a client certificate and if one is not presented from the client, NiFi will move on to next authentication method. 2. I am not clear what you mean by "removed excessive permissions from authorizations.xml".  If you are using Ranger, the authorizations.xml file is not being used.  That file would have been created by the file-access-policy-provider.  Ranger does not use this provider.  There really is no concept of an "initial admin" when using Ranger.  You'll need to add authorization for what you need manually in Ranger.  The "Initial Admin" is used when NiFi authorization is handled by a local file provider so that a user can be setup on startup that has ability to access NiFi and setup additional authorizations from within the NiFi UI.  I recommend starting a new community question so we don't make this thread overly complicated by solving many unrelated issues.   There are some NiFi Resource Identifiers that would only apply to file based authorization, but all other do work when used correctly. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click " Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Congratulations on resolving the issue and thanks for sharing the solution.  ... View more

Hi @pacman A Merge happens for each Read Operation i.e. BlockCache & MemStore. As such, Incorrect Values aren't observed. Having said that, If you observe any such scenario of Read/Write Inconsistency, Kindly share a Use-Case & any replication attempt to allow us to review accordingly.   Regards, Smarak ... View more

I have seen that you save the MapRecord as string. By mistake, i saved it also as string due to wrong schema. My record looks like this, any idea how can convert it back to MapRecord from this string format: "[MapRecord[{name=John Doe, age=21, products=[Ljava.lang.Object;@f8495e3, type=end-user, description=This is an end user}]]" Thanks! ... View more

This article could be improved if settings for the emitting and receving nifi instances could be more clearly identified. ... View more

@pacman  In addition to what @ckumar already shared: NiFI purposely leaves components visible to all on the canvas. but unless authorized to view those components, they will display as "ghost" implementations.   "Ghosted" components will not show any component names or classes on them.  They will only show stats.  Unauthorized users will be unable to view or modify the configuration.  User will also be unable to list or view data in connections (only see numbers of FlowFiles queued on a connection). The reason NiFi shows these ghosted components is to prevent multiple users from building their dataflows on top of one another.  It is very common for users to have multiple teams building their own dataflows, but then also have monitoring teams that may be authorized as "operators" across all dataflows. Or they may have some users that are members of multiple teams.   That means these users who can see more, would be left with potentially components layered on top of one another making management very difficult. The stats are there so even if a user can not view or modify a component, they can see where FlowFile backlogs are happening.  Since NiFi operates within a single JVM and every dataflow, no matter which user/team built them, is executed as the NiFi service user, everything must share the same system resources (various repos, Heap memory, disk  I/O, cpu, etc).   These stats provide useful information that one team can use to communicate to another team should resource utilization become an issue. NiFi's authorization model allows users to make very granular access decisions for every component.  Authorizations are inherited from the parent process group unless more granular policies are setup on a child component (processor, controller service, input/output port, sub-process group, etc..).  Hope this helps, Matt ... View more