@John T Was I able to address your question here on how this putTCP --> listenTCP flow is working? ... View more

@Felix Albani Thank you for your feedback... I have made the correction. ... View more

@Arun A K Once NiFi is secured access to any resource in NiFi requires authentication and authorization. It is not required but highly recommended. When you go to add a new policy in ranger, you need to provide the exact "Nifi Resource identifier". If The Ranger user is able to communicate with NiFi securely, it can retrieve that list of resources and the user can just select from the list: Otherwise, no list will be returned and you must know the exact string to enter. So you do not need to SSL enable Ranger to accomplish this. NiFi can still communicate with an non secured ranger to retrieve authorizations. But you can configure Ranger to talk to a secured NiFi to obtain the resource listing: As seen above, the keystore and truststore configured her in Ranger will be used to connect to the nifi resources rest api endpoint to retrieve that resource listing. The keystore and truststore must be owned by the ranger user. The PrivateKeyEntry in the keystore must be trusted by the target NiFI truststore and the truststore used here must be able to trust the server certificate returned by the Target Nifi instance. Thanks, Matt ... View more

@John T First, let's make sure we are on the same page with terminology. FlowFile --> A FlowFile is something very specific to NiFi. Each "FlowFile" consists of two parts, the FlowFile Attributes/Metadata and the FlowFile content. In most cases, processors do not send "FlowFiles", but only send the FlowFile's content. The postHTTP has a option property to (send as FlowFile) which will allow it to send a complete FLowFile to a ListenHTTP processor running on another NiFi instance. NiFi's Site-To-Site capability (uses Remote Process Group and Remote input/output ports) to send "FlowFiles" between Nifi instances. The only other method to send a "FlowFile" between NiFi instances is to use the MergeContent processor configured with a "Merge Format" that will package up the complete FlowFile (FlowFile Stream or FlowFile tar) in the content. This packaged FlowFile can then be sent to another NiFi and the UnPackContent processor can be used to separate the original FlowFile Content from FlowFile Attributes (load original content on content repo and original Attributes in to FlowFile repo) Simply increasing the batch size to a large value here will not guarantee the original content is re-created in destination FlowFile. Since content is streaming in to the socket buffer and NiFi's ListenTCP processor simply calls the channel Reader to ingest from the buffer line-by-line, you may end up with lines from one source's content in the same FlowFile as lines of content from a different source File. For a little more info on the NiFi ListenTCP based processor: https://community.hortonworks.com/articles/30424/optimizing-performance-of-apache-nifis-network-lis.html If you end goal is to send complete FlowFile content, consider using a different transfer method from putTCP and listenTCP. The listenTCP processor is designed in such a way it assumes each line being read from teh socket buffer is its own incoming message. Thank you, Matt ... View more

@Chad Ewing You could use curl to verify that the access token nifi rest-api end-point is returning a token to you. # curl 'https://<nifihostname>:<port>/nifi-api/access/token' --data 'username=<username>&password=<password>' -k This should return a long string of characters which is your token. Once you know above works, you can use the following to store a retrieved token in to a variable for easier use in subsequent curl rest-api calls: # token="$(curl 'https://<nifihostname>:<port>/nifi-api/access/token' --data 'username=<username>&password=<password>' -k)" That token can then be used in subsequent calls to the rest-api as follows: # curl 'https://<nifihostname>:<port>/nifi-api/flow/current-user' -H "Authorization: Bearer $token" -k The above for example would return JSON output like: {"identity":"<username or DN>","anonymous":false,"provenancePermissions":{"canRead":false,"canWrite":false},"countersPermissions":{"canRead":false,"canWrite":false},"tenantsPermissions":{"canRead":true,"canWrite":true},"controllerPermissions":{"canRead":true,"canWrite":true},"policiesPermissions":{"canRead":true,"canWrite":true},"systemPermissions":{"canRead":false,"canWrite":false},"restrictedComponentsPermissions":{"canRead":false,"canWrite":true}} With your browser, the exchange is not much different. When you login, you are hitting that same endpoint to get a JWT token. Your browser stores that token for subsequent requests sent to the same NiFi (such as loading the flow/canvas). When the token arrives in the subsequent requests it is checked against the server side equivalent for that token which will be stores in the "nifi-user-keys.h2.db". This can be found in the NiFi database repository directory. So make sure this file exists, is owned by same user running NiFi process, and you have not run out of disk space. If the token looks good and user authentication is valid for the resource being requested, you'll get a response. The login-identity-providers.xml file configuration for your ldap-provider dictates how long these tokens remain good for. Default expiration is 12 hours at which time the client will need to get a new token. Thank you, Matt ... View more

@Chad Ewing Is this a NiFi Cluster? Do you have an external load-balancer sitting in front of your NiFi URL? If so you need to make sure you have enable sticky sessions. A Token is only good for the node from which it was issued. So If subsequent requests to a load-balanced URL go to a different node the token your browser presents will not be known to that node. Thank you, Matt ... View more

@Jan Can we keep each post in community.hortonworks.com to a single question? The above is no longer an TLS handshake issue. It is an authorization issue. Please start a new question for this topic. Accepting an answer for this question will close this thread. Thank you, Matt ... View more

@Jan Why only one node? Are you installing both NiFi and MiNiFi on the same server? I would execute the 'hostname -f" command on both the server where NiFi is installed and the server wheer MiNiFi is/will be installed. Then use those full hostnames in the tls-tookit.sh script to create two keystores for use on these two instances ./tls-toolkit.sh standalone -n '<nifi.hostname>,<minifi.hostname>' -C 'CN=<nifiuser>, OU=NIFI' -P <truststore passsword> -S <keystorepassword> You will end up with a directory for each server being created that contains the needed TLS configuration info/files. If you want to include SAN to your new certificates, you will need to create each one at a time: ./tls-toolkit.sh standalone -n '<nifi.hostname>' -C 'CN=<nifiuser>, OU=NIFI' -P <truststore passsword> -S <keystorepassword> --subjectAlternativeNames '<nifi-ip-address>,<etc>' ./tls-toolkit.sh standalone -n '<minifi.hostname>' -P <truststore passsword> -S <keystorepassword> --subjectAlternativeNames '<minifi-ip-address>,<etc>' You should not need to any SAN entry if you generate your keystores using the actual hostnames assigned to your servers. Also make sure that the following properties have been set to the appropariate hostnames as well in the nifi.properties file: nifi.remote.input.host= nifi.web.https.host= Once your NiFi server is using the new keystore and truststore files, you can use the nifi server hostname in the RPG. Thanks, Matt ... View more