@Former Member    This should not be happening unless you are clearing NiFi state between restarts or NiFi is having a state issue.  When the listSFTP processor runs, it will record state so it does not list the same files more than once. If the Files being written to the SFTP server from which you are listing are not using dot rename as they are being written, it is possible your ListSFTP processor is listing them while they are still being written to.  Since state is based off timestamps, a file still being written to will have its timestamp updated between executions of the listSFTP processor.  This would result in the file being listed again. If the standard ssh dot rename method is not being used, you may need to "minimum File Age" property in listSFTP to make sure only files whose timestamp has not updated for X amount of time are listed.  This of course depends on your version of NiFi being used since this property was introduced as part of https://issues.apache.org/jira/browse/NIFI-5977 There is also the possibility if you are using NiFi 1.8 you are hitting this known bug: https://issues.apache.org/jira/browse/NIFI-5849   If you found a solution that worked, please tae a moment to accept the solution to resolve this community question.   Thank you, Matt ... View more

@svasi    Sounds like you should be using a RouteOnAttribute processor between your listFile and FetchFile processor so that only those FlowFIles you want to Fetch the content for are routed to the FetchFile processor. The listFile processor produces one FlowFile for each file that is returned during the listing execution.  The listFile processor writes a number of FlowFile attributes to the FlowFiles that are produced. The Filename attribute is what you will want to use when writing your NiFi Expression Language (EL) statement for your dynamically created properties added to the RouteOnAttribute processor. You can find a EL guide under "Help" under the NiFi global menu in the upper right corner of the NiFi UI.   Hope this helps, Matt ... View more

@Former Member  The routeOnAttribute processor expects you to add new dynamic properties.  Each of those dynamic properties you add becomes a new relationship on the processor to use for routing. Each new property's value is expected to be a NiFi Expression Language statement that results in a boolean true or false.  If based on the FlowFile being processed the expression resolves to "true" then the FlowFile will be routed to this new relationship (property name is used as relationship name).  If none of your dynamic properties resolve to true for a given FlowFile, that FlowFile is routed to the pre-existing unmatched relationship.   I recommend performing this routing prior to actually fetching the content, if the intent is to drop any of the FlowFiles.  No sense it using network or disk I/O if you have no need for some content. The ListFile processor create a number of FlowFile attributes on FlowFile including the file.LastModifiedTime. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.10.0/org.apache.nifi.processors.standard.ListFile/index.html   NiFi Expression Language guide is embedded under help accessible vi the NiFi global menu in the upper right corner of the NiFi UI.  It is also available here: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html Note:  Above links from Apache NiFi are valid for most current release.  The embedded docs included in your installed release will be most accurate for your release.   Hope this helps, Matt ... View more

@saivenkatg55    NiFi musty be secured before you can use any form of authentication and authorization.   Along with securing NiFi you can configure NiFi's login-identity-providers.xml to use the "ldap-provider" for user authentication.   Here are some useful links: Securing NiFi: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.1.1/nifi-authentication/content/nifi_authentication.html Setting up ldap-provider: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.1.1/nifi-security/content/ldap_login_identity_provider.html Setting up authorization: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.1.1/nifi-authorization/content/nifi_authorization.html Hope this helps, Matt ... View more

@girish6    Anything you can do via the NiFi UI, can also be done using the NiFi Rest-api.   Using the developer tools available ion most browsers is a great way to learn how the rest-api calls work.  You can have the developer tools UI open while you perform similar actions on the canvas and then capture those rest-api request directly from the developer tools (most offer an option to "save as curl"). It can be challenging to build dataflows via the UI in terms of placement.  While NiFi does not care if all your components are stacked upon one another at position 0,0, if you ever try to access the UI to modify a component that way later it would be nearly impossible as it would be one blob in the center of the canvas.   You can find the rest-api docs embedded in your NiFi installation by going to the Global Menu in the upper right corner of your NiFi UI and selecting Help. Hope this helps, Matt ... View more

 @Love-Nifi  Just wanted to add that as @jsensharma mentioned, NiFi will enforce TLS 1.2 as of Apache NiFi release version 1.2.0 but only for all inbound connections to NiFi.  NiFi can still support negotiating lower TLS version when making outbound connections in order to support older destination systems.  Those processor would use a sslContextService which can be configured to restrict what TLS version is used/allowed. ... View more