@Chad Ewing Awesome, Glad to hear I was able to help. Please Take a moment to click "accept" below the answer to close out this thread. ... View more

@John T Was I able to address your question here on how this putTCP --> listenTCP flow is working? ... View more

@Felix Albani Thank you for your feedback... I have made the correction. ... View more

@Arun A K Once NiFi is secured access to any resource in NiFi requires authentication and authorization. It is not required but highly recommended. When you go to add a new policy in ranger, you need to provide the exact "Nifi Resource identifier". If The Ranger user is able to communicate with NiFi securely, it can retrieve that list of resources and the user can just select from the list: Otherwise, no list will be returned and you must know the exact string to enter. So you do not need to SSL enable Ranger to accomplish this. NiFi can still communicate with an non secured ranger to retrieve authorizations. But you can configure Ranger to talk to a secured NiFi to obtain the resource listing: As seen above, the keystore and truststore configured her in Ranger will be used to connect to the nifi resources rest api endpoint to retrieve that resource listing. The keystore and truststore must be owned by the ranger user. The PrivateKeyEntry in the keystore must be trusted by the target NiFI truststore and the truststore used here must be able to trust the server certificate returned by the Target Nifi instance. Thanks, Matt ... View more

@John T First, let's make sure we are on the same page with terminology. FlowFile --> A FlowFile is something very specific to NiFi. Each "FlowFile" consists of two parts, the FlowFile Attributes/Metadata and the FlowFile content. In most cases, processors do not send "FlowFiles", but only send the FlowFile's content. The postHTTP has a option property to (send as FlowFile) which will allow it to send a complete FLowFile to a ListenHTTP processor running on another NiFi instance. NiFi's Site-To-Site capability (uses Remote Process Group and Remote input/output ports) to send "FlowFiles" between Nifi instances. The only other method to send a "FlowFile" between NiFi instances is to use the MergeContent processor configured with a "Merge Format" that will package up the complete FlowFile (FlowFile Stream or FlowFile tar) in the content. This packaged FlowFile can then be sent to another NiFi and the UnPackContent processor can be used to separate the original FlowFile Content from FlowFile Attributes (load original content on content repo and original Attributes in to FlowFile repo) Simply increasing the batch size to a large value here will not guarantee the original content is re-created in destination FlowFile. Since content is streaming in to the socket buffer and NiFi's ListenTCP processor simply calls the channel Reader to ingest from the buffer line-by-line, you may end up with lines from one source's content in the same FlowFile as lines of content from a different source File. For a little more info on the NiFi ListenTCP based processor: https://community.hortonworks.com/articles/30424/optimizing-performance-of-apache-nifis-network-lis.html If you end goal is to send complete FlowFile content, consider using a different transfer method from putTCP and listenTCP. The listenTCP processor is designed in such a way it assumes each line being read from teh socket buffer is its own incoming message. Thank you, Matt ... View more

@Chad Ewing You could use curl to verify that the access token nifi rest-api end-point is returning a token to you. # curl 'https://<nifihostname>:<port>/nifi-api/access/token' --data 'username=<username>&password=<password>' -k This should return a long string of characters which is your token. Once you know above works, you can use the following to store a retrieved token in to a variable for easier use in subsequent curl rest-api calls: # token="$(curl 'https://<nifihostname>:<port>/nifi-api/access/token' --data 'username=<username>&password=<password>' -k)" That token can then be used in subsequent calls to the rest-api as follows: # curl 'https://<nifihostname>:<port>/nifi-api/flow/current-user' -H "Authorization: Bearer $token" -k The above for example would return JSON output like: {"identity":"<username or DN>","anonymous":false,"provenancePermissions":{"canRead":false,"canWrite":false},"countersPermissions":{"canRead":false,"canWrite":false},"tenantsPermissions":{"canRead":true,"canWrite":true},"controllerPermissions":{"canRead":true,"canWrite":true},"policiesPermissions":{"canRead":true,"canWrite":true},"systemPermissions":{"canRead":false,"canWrite":false},"restrictedComponentsPermissions":{"canRead":false,"canWrite":true}} With your browser, the exchange is not much different. When you login, you are hitting that same endpoint to get a JWT token. Your browser stores that token for subsequent requests sent to the same NiFi (such as loading the flow/canvas). When the token arrives in the subsequent requests it is checked against the server side equivalent for that token which will be stores in the "nifi-user-keys.h2.db". This can be found in the NiFi database repository directory. So make sure this file exists, is owned by same user running NiFi process, and you have not run out of disk space. If the token looks good and user authentication is valid for the resource being requested, you'll get a response. The login-identity-providers.xml file configuration for your ldap-provider dictates how long these tokens remain good for. Default expiration is 12 hours at which time the client will need to get a new token. Thank you, Matt ... View more