Member since
07-30-2019
925
Posts
196
Kudos Received
91
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1152 | 10-05-2021 01:53 PM | |
15025 | 09-23-2019 06:03 AM | |
5895 | 05-04-2019 08:42 PM | |
1239 | 06-11-2018 12:45 PM | |
11106 | 06-04-2018 01:11 PM |
04-19-2017
05:44 PM
@Yogesh Sharma So, using ListSFTP retains state and will only add new files to the list when it is run, so the FetchSFTP will only pull each file once, where with GetSFTP, you would pull the same files over and over again.
... View more
04-19-2017
01:30 PM
2 Kudos
@Yogesh Sharma FetchSFTP is cluster friendly where as GetSFTP is not, meaning you can have multiple nodes in a cluster pulling data from the same source, but with GetSFTP, you cannot. If you have a NiFi cluster and you are using the GetSFTP processor, you would have to configure that processor to run on the primary node only so the other nodes in the cluster wouldn't try to pull the same files. This will result in the files only being pulled by a single node in a cluster instead of being pulled by all nodes, which is what FetchSFTP would do. Do you plan to leave the files on the remote system or pull them and remove them? Do you have a cluster or standalone instance of NiFi?
... View more
04-19-2017
01:21 PM
@TARANA POLEKAR Are the two instances of NiFi running on the same system and bound to different IPs?
... View more
04-19-2017
01:15 PM
1 Kudo
@Raj B So, your basic flow is the sending systems pushes data to the load balancer and then the load balancer does a round robin approach to each node in the NiFi cluster? What would the load balancer do if one of the nodes is down? Is it able to do automatic failover?
... View more
04-18-2017
09:31 PM
@Raj B Are you using SFTP to push data?
... View more
04-18-2017
07:13 PM
@Raj B Is it possible to have NiFi pull the data from the source?
... View more
04-18-2017
05:30 PM
@Raj B
Can java be installed on the load balancer? If so, I would install NiFi on the load balancer and then just add a Remote process group to the NiFi instance on the load balancer and let NiFi handle the load balancing.
... View more
04-18-2017
03:49 PM
1 Kudo
@Ed Prout One way around the issue, is to monitor the success relationship out of the GetSplunk processor using the MonitorActivity processor. If data does not pass through in a set time period, the the MonitorActivity processor generates an "inactive" flow file and this can be used as a trigger for an ExecuteScript processor which would run a curl script to restart the processor. Not an elegant solution, but it should work.
... View more
04-18-2017
03:33 PM
@Gu Gur You could you the ExecuteScript processor, but that would probably be more work than just doing it manually since it is only two nodes you are adding. I am assuming you have not secured the cluster. You could just copy the conf directory over to the new nodes, and then update the all entries to match the name of the new host. For example, assuming the current nodes are node1.localdomain, node2.localdomain and node3.localdomain, then copy node3's conf directory to a new node you want to add, you could just change everywhere you have node3 to node4, or whatever is the new node system name. Are you following me?
... View more
04-18-2017
01:45 PM
2 Kudos
@Harish S Add the line "useTicketCache=true" to the jaas.conf file. What are the differences between the dev and test environments? Are you using the same KDC server? Same kafka brokers?
... View more