HOW CAN I SET AN INPUT/OUTPUT PORTS FOR A REMOTE PROCESS GROUP ?

[#1]

Hello everyone thanks for reading me,

i'm very new to nifi as you can see in some of my previous question 🙂

I have an unsecure NIFI 1.0 cluster, i think i have configured the nifi.properties and the zookeeper.properties right

but when i'm trying to use a REMOTE PROCESS GROUP , i always end up with this " NiFi Flow does not have any input ports "

This is my nifi.properties for one of my three servers

# Site to Site properties nifi.remote.input.host=server1 nifi.remote.input.secure=false nifi.remote.input.socket.port=8022 nifi.remote.input.http.enabled=true nifi.remote.input.http.transaction.ttl=30 sec

# web properties nifi.web.war.directory=./lib nifi.web.http.host=server1 nifi.web.http.port=8070 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200

# cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=server1 nifi.cluster.node.protocol.port=8021 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file= nifi.cluster.flow.election.max.wait.time=5 mins nifi.cluster.flow.election.max.candidates=3

I have tried all configuration possible of the RPG with all the port here but I can't connect the RPG to a get file processor,

I have read this https://community.hortonworks.com/articles/16461/nifi-understanding-how-to-use-process-groups-and-r....

and this https://community.hortonworks.com/questions/47516/nifi-rpg-and-clustering.html

but those are for nifi inferiror version , can anyone explain me how to use the RPG without a NCM ?

What i want to do is to get some file localy on one of my Nifi node by s2s but i think i'm missing something here

Thanks for your time

[#2]

@Toky Raobelina

Are the configs provided above from the system you are trying to send data to? Did you add an "input port" to the root (top level) if the canvas on that server? So I am gathering you were able to add the RPG, but it is reporting "no input ports available"?

Matt

[#3]

Question 1 : yes this configs are from the system i'm trying to send data ( for the record all my system with nifi have this configs)

Question 2 : i don't undrstand your question

Question 3 : you are correct i was able to enable the transmision but when i go to remote ports idon't find any

Toky

[#4 · Accepted solution]

@Toky Raobelina

When you add an RPG to the graph, the URL you provide in its configuration should be the same URL you would use to access that target NiFi's UI. You will then select whether to use RAW or HTTP protocols.

With RAW (legacy format) your source NiFi will send data across port 8022. With HTTP protocol the Source NiFi will send data across port 8070. So you need to make sure with RAW that your source NiFi can communicate with port 8022 on the target NiFi.

Based on what you shared, it sounds more like you have set up an entry point on the target NiFi to accept the data you want to send to it.

On the target NiFi you will need to add an input port to your canvas. This port must be added at the root level to be used for S2S. BY root level, i mean it cannot be contained within a process group on the target NiFi.

So on the canvas of your target NiFi, you will have a flow that looks similar to this:

It may take a couple minutes for the input port to show up on the RPG located on the Source NiFi. You can always right click on the RPG and select "Refresh".

On your Source NiFi you will have a flow similar to this:

Once the input port is set to run and you have enabled transmission on your RPG, data should start flowing unless a firewall is blocking the connection on part 8022 as mentioned above.

Thanks,

Matt

[#5]

@Toky Raobelina

Did the information provided help resolve your S2S issue?

[#6]

Thanks Matt for your help sorry i' can't test it right now ( it's a bit late where i am) but when i''ve done it i'will inform you right Away . Just To be clear i get it right, in the option URL i put the URL of the target and set an input ports on The Ui of the target and then this input Will appear on the RPG (remote ports) If i choose raw it will be on port 8022 and 8070 if i choose http. Feel free to correct me if i made mistake.

Toky

[#7]

I think you have it.

[#8]

Hello @Matt

Thanks for your precious help, it's working just fine 🙂

[#9]

Hello @Matt

Just one more question, i have taileFile ==> RPG,

the taileFiile is working but all the data are queued and doesn't pass through the RPG, do you know why ?

Thanks

Toky