@Francisco Pires What about in front of your directory property value "C:\SNIRH"? Can you attach a template of this configured processor? ... View more

@Gaurav Jain This is the exact use case for why GetSFTP was deprecated in favor of listSFTP and FetchSFTP processors. The ListSFTP processor would run on the primary node only. It produces one 0 byte FlowFile for every file in the listing. All these 0 byte FlowFiles are then sent to a RPG for distribution across cluster. The distributed files are then fed to a FetchSFTP processor that will retrieve the content form the SFTP server and insert it in to the FlowFile at that time. This model eliminates the overhead on the Primary node since it does not need to write the content and it reduces network overhead between nodes since their is no content being send in FlowFiles via the RPG. The only issue you are going to run in to is: https://issues.apache.org/jira/browse/NIFI-1202 This issue is addressed in Apache NiFi 1.2.0 which was just released this week. It will also be addressed in HDF 3.0 which will be released soon. You can work around the issue in older versions by setting a small object backpressure threshold on the connection feeding your RPG. Since this backpressure is a soft limit, you need to put a processor between your listSFTP processor and the RPG that only processes FlowFiles one at a time. I recommend RouteOnAttribute (no configuration needed on processor, simply route the one existing "unmatched" relationship to the RPG and set back pressure on that connection). Thanks, Matt ... View more

Based on what you provided above it looks like you installed via the nifi tar.gz file. By default NiFi runs unsecure on port 8080. So yes, you want to change that port to some unused port on your server. Thanks, Matt ... View more

@Anthony Murphy NiFi stores state differently depending on whether your NiFi is installed as cluster or a standalone instance. With a cluster, NiFi stores state in zookeeper. As long as your new Nifi points to the same zookeeper or the zookeeper content has been moved to a new zookeeper you are using, state will be preserved. In a Standalone Nifi install, state is recorded on disk. You can look in your NiFi's state-management.xml file to see/change the configuration of both the "local-provider" (used by standalone NiFi) or "cluster-provider" (used by Clustered NiFi). You van change where state is being written to here. Thanks, Matt ... View more

@umair ahmed I am not a Windows Admin and could not tell you off teh top of my head. ... View more

@Gaurav Jain You can build into your dataflow the ability to redistribute FlowFiles between your nodes. Below are just some of the benefits NiFi clustering provides: 1. Redundancy - You don't have a single point of failure. You dataflows will still run even if a node is down or temporarily disconnected form your cluster. 2. Scaleable - You can scale out the size of your cluster to add additional nodes at any time. 3. Ease of Management - Often times a dataflow or multipole dataflows are constructed within the NiFi canvas. The volume of data may eventually push the limits of your hardware necessitating the need for additional hardware to support the processing load. You could stand up another standalone Nifi instance running the same dataflow, but then you have two separate dataflows/canvases you need to manage. Clustering allows you to make a change in only one UI and have those changes synced across multiple servers. 4. Provides Site-To-Site for load-balanced data delivery between NiFi end-points. As you design your dataflows, you must take in to consideration how the data will be ingested. - Are you running a listener of some sort on every node? In that case source systems you push data to your cluster through some external load-balancer. - Are you pulling data in to your cluster? Are you using a cluster friendly source like JMS or Kafka wheer multiple NiFi nodes can pull data at the same time? Are you using non-cluster friendly protocols to pull data like SFTP or FTP? (In case like this load-balancing should be handled through list<protocol> --> RPG Input port --> Fetch<protocol> model) NiFi has data HA on its future roadmap which will allow other nodes to pickup work on data of a down node. Even when this is complete, I do not believe it will doing any behinds the scenes data redistribution. Thanks, Matt ... View more

I literally hit the "tab" key on my keyboard. ... View more