Hi guys, It seems Repository has to be configured as a path to local disk. Could we specify the content repository to S3 or any other remote location(hdfs), etc? Thanks. ... View more

Hi guys, I am confused in a couple of questions on NCM failure. My understanding is the main responsibilities of NCM are: •Communicates dataflow changes to the nodes •Receives health and status information from nodes If NCM fails, the existing dataflow can still run, but can't be changed. And new node can't join in, dead node can't be detected. How about we have below two scenarios: 1. A ListSftp processor deployed in Primary node, it will distribute filepaths to each work nodes running FetchSftp. However, if NCM fails, and one of the work nodes also fails, will the primary node still send data to the dead work node? How does primary node know that node is dead? 2. If we setup "DistributedMapCacheServer" in NCM, later NCM fails, does that mean the work node can't access to the Cache Server any longer? Any solution to make it high available? Thanks. ... View more

I have two NiFi instances deployed in AWS. I'd like to test the RGP across clusters US-East: 172.31.48.1 Dataflow: RGP->PutFile # Site to Site properties nifi.remote.input.socket.host=172.31.48.1 nifi.remote.input.socket.port=9090 nifi.remote.input.secure=false # web properties # nifi.web.http.host=172.31.48.1 nifi.web.http.port=8080 Singapore: 172.31.11.2 Dataflow: GetFile->output_port # Site to Site properties nifi.remote.input.socket.host=172.31.11.2 nifi.remote.input.socket.port=9090 nifi.remote.input.secure=false # web properties # nifi.web.http.host=172.31.11.2 nifi.web.http.port=8080 The GetFile processor can fetch files and queue them in Singapore instance. The RGP can successfully detect the "output_port" by setting the url as "http://sig-public-ip:8080/nifi" in US-east instance. However, RGP log shows : 14:38:14 UTC ERROR e93c67c6-3472-44e4-aa3e-78613ce37e47 us-ease-public-ip:8080 RemoteGroupPort[name=output_port,target=http://sig-public-ip:8080/nifi] failed to communicate with http://sig-public-ip:8080/nifi due to java.net.SocketTimeoutException When I test the network connection, ubuntu@ip-172-31-48-1:/opt$ telnet sig-public-ip 9090 Trying sig-public-ip... Connected to sig-public-ip. ubuntu@ip-172-31-11-2:/opt$ telnet us-ease-public-ip 9090 Trying us-ease-public-ip... Connected to us-ease-public-ip. The same for port 8080. Any settings I should do to enable the RPG communication? Thanks. ... View more