Nifi 1.1 site-to-site permissions are disabled within the GUI

[#1]

Hi,

I'm trying to configure a site-to-site connection from a Minifi to a Nifi (v1.1).

When I'm trying to set the "Process" Input port "site-to-site" policies, I can't select it ! See the attachment, it is disabled !

But I can modify the site-to-site "Access policies".

I'm logged in with an admin account.

I don't know why ! Do I miss some Nifi configuration ?

Olivier

[#2]

@Olivier Drouin

Where is this input port "From Minifi" located on your canvas? At top level canvas or with a process group?

Input and output ports facilitate the movement of FlowFiles between a parent process group and a child process group (where the input and output ports exist). So Input/output ports created within a process group only allow for receiving or sending of FlowFiles from the parent process group.

The top level of your canvas is actually just another process group (root process group). When you create an input or output port here it becomes a remote input/output port capable of transferring Flowfiles over S2S. You will even see that these root level input/output ports are rendered a little different and have more configuration options.

Thanks,

Matt

[#3]

@Matt Clarke

Hello Matt

I'm working with Olivier on this Site-To-Site configuration. In fact, purpose is to create a Nifi graph to export it to MiNiFi, this graph will send file/flow from MiNiFi to NiFi. Treatment on NiFi will be to ingest the flow over Kafka.

Nifi is configured for Site-To-Site, and I just want to create a graph to read a local file and push it to the remote processor group.

I've tryed to summarize in a small picture the situation where we reproduce the rights issue as mentioned by Olivier.

Thank in advance for your help

[#4]

@Olivier Drouin and @Xavier COUDRE

-

At this time Remote Process Groups are only able to see "remote" input and output port. NiFi only treats input and output ports added to the root canvas of the target NiFi as remote Process groups.

So as you can see from the above diagram, your MiNiFi RPG will only be able to see the input port at the root level. You will then need to use an input port in each sub-process group to move the received FlowFiles from the root canvas level down in to Process group level 3. I completely understand the complexity this adds in a multi-nested process group dataflow design.

-

There is a Jira out there requesting an enhancement to NiFi to separate input/output ports into local and remote: components. https://issues.apache.org/jira/browse/NIFI-2933
-
However, my understanding is that there are significant design hurdles for such a change. There have been other design discussion around being able to connect directly from a top level process group to a deep nested process group without needing to create ports at each level.

-

Hope this helps to clarify things,

Matt

[#5]

@Matt Clarke

Hello Matt

Thanks for your prompt answer

I've implemented the input as above, and I was able to add the proper rights to receive flow.

BUT, I still have the same issue on remote processor group: any idea ?

2018-04-25 10:18:47,997 WARN [Remote Process Group 01611143-1fd7-134e-cf08-9f757f9dcf71: https://obitedhs-vet001.data.edh:9091/ Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://obitedhs-vet001.data.edh:9091/nifi-api due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 401:Unauthorized with explanation: null
2018-04-25 10:18:48,083 ERROR [Remote Process Group 01611143-1fd7-134e-cf08-9f757f9dcf71: https://obitedhs-vet001.data.edh:9091/ Thread-1] o.a.n.remote.StandardRemoteProcessGroup org.apache.nifi.remote.StandardRemoteProcessGroup$InitializationTask@d6b60fb Failed to request account: got unexpected response code of 404:Not Found

Thanks and best regards

[#6]

@Xavier COUDRE
There are multiple policies needed for S2S to work.

The first policy is a "Global" policy (Found under "Policies" via the hamburger menu in upper right corner of NiFi UI):

You must authorize the source NiFi instance(s) (This would be any NiFi instance running a RPG trying to connect to this NiFi). This allows the RPG to communicate with the target to retrieve details such as number of connected nodes, load on those nodes, list of authorized input and output ports and supported connection information (node hostnames, is RAW transfer enabled, what is RAW port...)

-

In order for any ports to be returned, the source NiFi instance(s) must be authorized in those ports. I believe this is the step you have already completed which we were discussing initially in this answer.

-

Thanks,

Matt

-

HCC Forum Tip: Try to avoid responding to an existing answer with a new answer. Comment on answer instead. It makes discussion easier to follow, especially when multiple answer are being discussed.

[#7]

@Xavier COUDRE

Couple things here....

1. None of the components in NiFi (processors, remote process groups, ports, etc...) are running as the user who is authenticated in to the NiFi UI. So authorizing users on input/output ports will have no affect here.

2. The source NiFi/MiNIFi must be configured with SSL certificates. NiFi's Site-To-Site protocol uses SSL to authenticate between NiFi instances.

3. Your screenshot below shows you authorizing what appears to be a user and not the source NiFi instance(s) to a Input port on the root canvas level. But did you also authorize the source NiFi instance(s) for the "GLOBAL" "retrieve site-to-site details" policy:

As you can see by the above, I authorized my 3 NiFi instances in my cluster to retrieve site-to-site details.

-

And then on my input port itself, I authorized my 3 NiFi instances access to the input port:

-

Hope this helps,

Matt

[#8]

@Matt Clarke

Hello

Rights are configured at "root" input level: should we restart Nifi to apply new rights ?

[#9]

@Olivier Drouin @Xavier COUDRE

Did you get your Site-To-Site working, If you found the answer below helpful, please take a moment to login and click "accept" below the answer.