@John T Was I able to address your question here on how this putTCP --> listenTCP flow is working? ... View more

@Felix Albani Thank you for your feedback... I have made the correction. ... View more

@Arun A K Once NiFi is secured access to any resource in NiFi requires authentication and authorization. It is not required but highly recommended. When you go to add a new policy in ranger, you need to provide the exact "Nifi Resource identifier". If The Ranger user is able to communicate with NiFi securely, it can retrieve that list of resources and the user can just select from the list: Otherwise, no list will be returned and you must know the exact string to enter. So you do not need to SSL enable Ranger to accomplish this. NiFi can still communicate with an non secured ranger to retrieve authorizations. But you can configure Ranger to talk to a secured NiFi to obtain the resource listing: As seen above, the keystore and truststore configured her in Ranger will be used to connect to the nifi resources rest api endpoint to retrieve that resource listing. The keystore and truststore must be owned by the ranger user. The PrivateKeyEntry in the keystore must be trusted by the target NiFI truststore and the truststore used here must be able to trust the server certificate returned by the Target Nifi instance. Thanks, Matt ... View more

@John T First, let's make sure we are on the same page with terminology. FlowFile --> A FlowFile is something very specific to NiFi. Each "FlowFile" consists of two parts, the FlowFile Attributes/Metadata and the FlowFile content. In most cases, processors do not send "FlowFiles", but only send the FlowFile's content. The postHTTP has a option property to (send as FlowFile) which will allow it to send a complete FLowFile to a ListenHTTP processor running on another NiFi instance. NiFi's Site-To-Site capability (uses Remote Process Group and Remote input/output ports) to send "FlowFiles" between Nifi instances. The only other method to send a "FlowFile" between NiFi instances is to use the MergeContent processor configured with a "Merge Format" that will package up the complete FlowFile (FlowFile Stream or FlowFile tar) in the content. This packaged FlowFile can then be sent to another NiFi and the UnPackContent processor can be used to separate the original FlowFile Content from FlowFile Attributes (load original content on content repo and original Attributes in to FlowFile repo) Simply increasing the batch size to a large value here will not guarantee the original content is re-created in destination FlowFile. Since content is streaming in to the socket buffer and NiFi's ListenTCP processor simply calls the channel Reader to ingest from the buffer line-by-line, you may end up with lines from one source's content in the same FlowFile as lines of content from a different source File. For a little more info on the NiFi ListenTCP based processor: https://community.hortonworks.com/articles/30424/optimizing-performance-of-apache-nifis-network-lis.html If you end goal is to send complete FlowFile content, consider using a different transfer method from putTCP and listenTCP. The listenTCP processor is designed in such a way it assumes each line being read from teh socket buffer is its own incoming message. Thank you, Matt ... View more