Looks like security zone is also working with nifi when configuring with ranger provided all the resource identifiers that needs to be added is also added in the security zone that we are creating. Probably this is a basic that I left out.  Currently the resources identifiers in policies as shown below.   But the security zone has only /flow as shown below     The policies json corresponding to the above configuration is shown below. {"serviceName":"nifi","serviceId":1,"policyVersion":140,"policyUpdateTime":"20230921-12:51:41.126-+0530","policies":[],"serviceDef":{"name":"nifi","displayName":"nifi","implClass":"org.apache.ranger.services.nifi.RangerServiceNiFi","label":"NIFI","description":"NiFi","options":{"enableDenyAndExceptionsInPolicies":"false"},"configs":[{"itemId":400,"name":"nifi.url","type":"string","mandatory":true,"defaultValue":"http://localhost:8080/nifi-api/resources","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"The URL of the NiFi REST API that provides the available resources.\"}","label":"NiFi URL"},{"itemId":410,"name":"nifi.authentication","type":"enum","subType":"authType","mandatory":true,"defaultValue":"NONE","label":"Authentication Type"},{"itemId":411,"name":"nifi.ssl.use.default.context","type":"bool","subType":"YesTrue:NoFalse","mandatory":true,"defaultValue":"false","uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"If true, then Ranger\u0027s keystore and truststore will be used to communicate with NiFi. If false, the keystore and truststore properties must be provided.\"}","label":"Use Ranger\u0027s Default SSL Context"},{"itemId":500,"name":"nifi.ssl.keystore","type":"string","mandatory":false,"label":"Keystore"},{"itemId":510,"name":"nifi.ssl.keystoreType","type":"string","mandatory":false,"label":"Keystore Type"},{"itemId":520,"name":"nifi.ssl.keystorePassword","type":"password","mandatory":false,"label":"Keystore Password"},{"itemId":530,"name":"nifi.ssl.truststore","type":"string","mandatory":false,"label":"Truststore"},{"itemId":540,"name":"nifi.ssl.truststoreType","type":"string","mandatory":false,"label":"Truststore Type"},{"itemId":550,"name":"nifi.ssl.truststorePassword","type":"password","mandatory":false,"label":"Truststore Password"},{"itemId":560,"name":"ranger.plugin.audit.filters","type":"string","mandatory":false,"defaultValue":"[]","label":"Ranger Default Audit Filters"}],"resources":[{"itemId":100,"name":"nifi-resource","type":"string","level":10,"mandatory":true,"lookupSupported":true,"recursiveSupported":false,"excludesSupported":false,"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher","matcherOptions":{"wildCard":"true","ignoreCase":"true"},"label":"NiFi Resource Identifier","description":"NiFi Resource","accessTypeRestrictions":[],"isValidLeaf":true}],"accessTypes":[{"itemId":100,"name":"READ","label":"Read","impliedGrants":[]},{"itemId":200,"name":"WRITE","label":"Write","impliedGrants":[]}],"policyConditions":[{"itemId":1,"name":"_expression","evaluator":"org.apache.ranger.plugin.conditionevaluator.RangerScriptConditionEvaluator","evaluatorOptions":{"ui.isMultiline":"true"},"uiHint":"{ \"isMultiline\":true }","label":"Enter boolean expression","description":"Boolean expression"}],"contextEnrichers":[],"enums":[{"itemId":1,"name":"authType","elements":[{"itemId":1,"name":"NONE","label":"None"},{"itemId":2,"name":"SSL","label":"SSL"}],"defaultIndex":0}],"dataMaskDef":{"maskTypes":[],"accessTypes":[],"resources":[]},"rowFilterDef":{"accessTypes":[],"resources":[]},"id":10,"guid":"a23a32e1-4b74-4cbf-8b60-3e161d748954","isEnabled":true,"createTime":"20230912-17:43:16.455-+0530","updateTime":"20230912-17:43:16.798-+0530","version":1},"auditMode":"audit-default","securityZones":{"nifi":{"zoneName":"nifi","resources":[{"nifi-resource":["/flow"]}],"policies":[{"service":"nifi","name":"nifi-zone-policy","policyType":0,"policyPriority":0,"isAuditEnabled":true,"resources":{"nifi-resource":{"values":["/proxy"],"isExcludes":false,"isRecursive":false}},"additionalResources":[{"nifi-resource":{"values":["/flow"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}},{"nifi-resource":{"values":["/data/process-groups/b2cf12da-018a-1000-9dd7-6e78a33341fb"],"isExcludes":false,"isRecursive":false}}],"policyItems":[{"accesses":[{"type":"READ","isAllowed":true},{"type":"WRITE","isAllowed":true}],"users":["vishnu"],"groups":[],"roles":["ranger_users"],"conditions":[],"delegateAdmin":true}],"denyPolicyItems":[],"allowExceptions":[],"denyExceptions":[],"dataMaskPolicyItems":[],"rowFilterPolicyItems":[],"serviceType":"nifi","options":{},"validitySchedules":[],"policyLabels":[],"zoneName":"nifi","isDenyAllElse":false,"id":4,"guid":"b4577f50-f5fc-40ae-b745-7ec9380e2705","isEnabled":true,"version":13}],"containsAssociatedTagService":false}}} Since only flow was in the security zone the other configurations was not working. If I add other two resource specifiers in the nifi security zone, then then things are working properly. Also not sure if there is some other thing that you mentioned as security zone. ... View more

@pacman  1. There is noway to disable TLS. If you remove your TLS certificate from your browser or use and incognito window your client certificate will not be presented in the TLS exchange. NiFi requires TLS certificate authentication for NiFi to NiFi authenticated and authorized connections (for example in a multi-node NiFi cluster or utilizing NiFi Site-to-Site between different NiFi deployments).  When NiFi is secured an NO other methods of authentication are configured, NiFi will "REQUIRE" a MutualTLS exchange.  Once at least one additional method of authentication is  configured, NiFi will "WANT" a client certificate and if one is not presented from the client, NiFi will move on to next authentication method. 2. I am not clear what you mean by "removed excessive permissions from authorizations.xml".  If you are using Ranger, the authorizations.xml file is not being used.  That file would have been created by the file-access-policy-provider.  Ranger does not use this provider.  There really is no concept of an "initial admin" when using Ranger.  You'll need to add authorization for what you need manually in Ranger.  The "Initial Admin" is used when NiFi authorization is handled by a local file provider so that a user can be setup on startup that has ability to access NiFi and setup additional authorizations from within the NiFi UI.  I recommend starting a new community question so we don't make this thread overly complicated by solving many unrelated issues.   There are some NiFi Resource Identifiers that would only apply to file based authorization, but all other do work when used correctly. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Kumar_  The more detail you can provide, the better chance with getting a response. Sharing details like: Version of NiFi being used. full error and stack trace logged in the nifi-app.log. processor configuration details (if applicable). Matt ... View more

@Anderosn  So MergeContent does just that, merges the content of all FlowFiles being merged. I am not sure how often your GenerateFlowFile processor executes, but when it does execute it will create a FlowFile with a unique filename (unless you set the filename in the GenerateFlowFile processor via a dynamic property).  The produced data by the GenerateFlowFile is routed is routed as a FlowFile to one of the success relationships and a clone FlowFile is routed to the other success relationship in your dataflow (both FlowFiles have same "filename" but different flowfile uuids). The "filename" attribute can be used in the MergeContent processor in the "Correlation Attribute Name" property.  Then you can set min num entries to "2".  This will make sure both FlowFiles with same value in the filename attribute will get allocated to same bin.  The MergeContent property "Attribute Strategy" will need to be set to "Keep All Unique Attributes" so that the final merged FlowFile will include the new token attribute. Now we have to deal with the content.  What we need to make sure is that the FlowFile used to fetch the token has no content before being routed to mergeContent processor. For that you can use the ModifyBytes processor and set "Remove all content" to "true" after your EvaluateJsonPath processor.  Removing the content does not remove the FlowFile metadata/attributes, so this now 0 byte FlowFile will still have its filename value and token attribute with value. ------- Now with above suggestion for your existing dataflow as an option, there are probably many other dataflow designs to accomplish this. Since you are using GenerateFlowFile to create the content needed for your final invokeHTTP, I'd go a different route that does not need a MergeContent processor. GenerateFlowFIle (custom content needed to fetch token) --> InvokeHTTP (get Token) --> EvaluateJsonPath (extract token from content to attribute) --> replaceText ( ("Replacement Strategy"="always replace", "Evaluation mode"="Entire text", "replacement value"=<content needed for your final rest-api call>) --> InvokeHTTP (you final rest-api endpoint request). The above removes need for MergeContent or dealing with multiple paths.  You have a single process flow where any failure in along the path does not result in potential of orphaned binned FlowFile at your MergeContent processor. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Kumar_  Check the configuration value boxes to see if you added a line return. If the configuration pop-up box shows more then just a "1" line then you have a lone return at the end of line "1".  Click on line to and hit delete to get rid of the line return if present. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@LKB  I'd expect a full stack trace in the nifi-app.log following that ERROR. The ldap-provider is configured inside the login-identity-providers.xml file.  Does your LDAP Manager password happen to contain any XML special characters?  If so, you should escape them. https://xml.silmaril.ie/specials.html Have you tried using ldapsearch to manually make a connection to your LDAPS from the server where you have NiFi installed using same manager DN, Manager password, and keystore and trustsore?  Was it successful? If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@manishg  The ListFile does not pickup any files.  It simply generates a zero content NiFI FlowFile for each file found in the target directory.  That FlowFile only has metadata about the target content.  The FetchFile processor utilizes that metadata to fetch that actual content and add it to the FlowFile.  The value added here happens when you have a lot target files to ingest.  To avoid having all the disk I/o related to that content on one node, you can redistribute the zero byte FlowFiles across all nodes so that each node now in a distributed way fetches the content (This works assuming that same target directory is mounted on all NiFi cluster nodes).  As @SAMSAL shared you could use Process Group (PG) FlowFile concurrency to accomplish the processing of one FlowFile at a time. The ListFile will still continue to list all FlowFiles in target directory (writes state and continues to list new files as they get added to input directory).  You can then feed the outbound connection of your ListFile to a PG configured with "Single FlowFile Per Node" FlowFile concurrency.  This will prevent any other FlowFile queued between ListFile and the PG to enter the PG until the first FlowFile has processed through that PG.   So your first processor inside the PG would be your FetchFile processor.   Now if you were to configure Load Balanced Connection on that connection between ListFile and the PG, You would end up with each node in your NiFi cluster processing a single File at a time.  This gives you some concurrency if you want it.  However, if you have a strict one file at a time, you would not configure load balanced connection. Hope this helps, Matt ... View more

 > ListFile will lose tracking. This is beyond my knowledge. According to the introduction of ListFile, the tracking entity will be stored in redis. Why is it lost when I only upgrade nifi? ... View more