Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

Solved Go to solution
Highlighted

Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

New Contributor

Hi All,

Iam trying to connect 2 NIFI Instances where one NIFI instance is secure (https ) and other is not ( http ).

I'am able to connect from https -> http and pull data but i would like to try the reverse pattern from http --> https.

My questions are:

1. How do or should i specify user certificate credentials in Remote Process group to connect to https NIFI instance.

2.By setting "nifi.remote.input.secure" to " false" on the secured NIFI instance,will i be able to connect.

Please guide or advice

Thanks in adv,

Time

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

I just tried this and realized I mis-spoke about setting "nifi.remote.input.secure" to false on the https instance and not needing the certs on the http instance. The reason why is because the http instance still needs to connect to the https instance initially to ask for the value of "nifi.remote.input.port". So even though the resulting site-to-site connection would be unsecure, the initial connection still has to be secure.

Here is my best attempt to recreate the steps I just followed that ended up working...

I created two copies of nifi-0.6.0-SNAPSHOT in a directory and called one nifi-https and one nifi-http.

On nifi-https I configured the following properties in nifi.properties (everything else left as defaults):

nifi.remote.input.socket.host=hostname.from.my.cert
nifi.remote.input.socket.port=8899
nifi.remote.input.secure=true

nifi.web.http.port=
nifi.web.https.port=8443

nifi.security.keystore=mycert.p12
nifi.security.keystoreType=PKCS12
nifi.security.keystorePasswd=mypassword
nifi.security.truststore=mytruststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=mypassword 

That makes nifi-https a secured instance with the web UI running on 8443 and a secure site-to-site connect available on 8899.

Now I went to https://localhost:8443/nifi in my browser and got prompted to request an account. At this point I edited nifi.properties again to turn on regular http access by setting nifi.web.http.port=8080, restarted nifi, went to http://localhost:8080/nifi and approved my account, then removed http access and restarted again, and was able to access over https. I then created a an Output Port called "Test" with a GenerateFlowFile sending data to it.

At this point nifi-https is fully setup.

On nifi-http I configured the following properties in nifi.properties (everything else left as defaults):

nifi.security.keystore=mycert.p12
nifi.security.keystoreType=PKCS12
nifi.security.keystorePasswd=mypassword
nifi.security.truststore=mytruststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=mypassword

That makes nifi-http a regular unsecured instance running on port 8080, but it now has the cert and truststore to make outbound secure connections.

Now I went to http://localhost:8080/nifi and create a Remote Process Group (RPG) with a URL of https://hostname.from.my.cert:8443/nifi. It is import that the hostname in this URL matches the value of "nifi.remote.input.socket.host" from the nifi-https instance.

Now I right-clicked on the RPG and chose Enable Transmission at which point I got a message that an account was requested. This happened because nifi-http is using mycert.p12 to connect to nifi-https, but nifi-https does not have an approved account for mycert.p12.

So I went to nifi-https (https://localhost:8443/nifi) and went to the accounts section and approved the account for mycert.p12 and chose a role of "NiFi'.

We also need to give the mycert.p12 user access to the "Test" Output Port. So on the https instance I stopped "Test", right-clicked and Configure, and from the Access Controls tab started typing the DN from mycert.p12, added that user to the Allowed Users list, hit Apply and started the port again.

Then I went back to nifi-http and right-clicked on the RPG and chose Refresh which caused it to retrieve the available Output Ports from nifi-https. I then connected the "Test" Output Port from the RPG to LogAttribute, started everything and it was able to pull FlowFiles from nifi-https.

View solution in original post

5 REPLIES 5
Highlighted

Re: Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

Hello,

There a couple of factors at play here...

Site-to-Site uses the the same SSL configuration that is also used to configure the SSL for the UI. This is provided through nifi.properties:

nifi.security.keystore= 
nifi.security.keystoreType= 
nifi.security.keystorePasswd= 
nifi.security.keyPasswd= 
nifi.security.truststore= 
nifi.security.truststoreType= 
nifi.security.truststorePasswd=

So you should be able to have an http instance, meaning the UI is not configured with a secure https port, but you still configure the keystore/truststore properties above, and it will use those to connect to the secure NiFi instance.

Secondly, on your https instance, if you set "nifi.remote.input.secure" to false then you should also be able to make a connection from from your http to https instance without configuring the above properties, but the connection will be unsecured in this case.

Highlighted

Re: Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

New Contributor

Hello,

Below was tried

[http->https]

1.http://<ip_address>:<port>/nifi

When tried to connect with "nifi.remote.input.secure" as "false" , there is "Unexpected end of file from server" Error Message on the http instance.

2.https://<ip_address>:<port>/nifi

When tried to connect with "nifi.remote.input.secure" as "false" , there is "No subject alternative names present" Error Message on the http instance.

3.[https<->https]

When tried to connect with "nifi.remote.input.secure" as "true" , there is "PKIX path validation failed: java.security.cert.CertPathValidatorException: signature check failed" Error Message on the both instance when tried either way.

Can any help me on this.

Thanks in adv,

Time

Highlighted

Re: Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

I just tried this and realized I mis-spoke about setting "nifi.remote.input.secure" to false on the https instance and not needing the certs on the http instance. The reason why is because the http instance still needs to connect to the https instance initially to ask for the value of "nifi.remote.input.port". So even though the resulting site-to-site connection would be unsecure, the initial connection still has to be secure.

Here is my best attempt to recreate the steps I just followed that ended up working...

I created two copies of nifi-0.6.0-SNAPSHOT in a directory and called one nifi-https and one nifi-http.

On nifi-https I configured the following properties in nifi.properties (everything else left as defaults):

nifi.remote.input.socket.host=hostname.from.my.cert
nifi.remote.input.socket.port=8899
nifi.remote.input.secure=true

nifi.web.http.port=
nifi.web.https.port=8443

nifi.security.keystore=mycert.p12
nifi.security.keystoreType=PKCS12
nifi.security.keystorePasswd=mypassword
nifi.security.truststore=mytruststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=mypassword 

That makes nifi-https a secured instance with the web UI running on 8443 and a secure site-to-site connect available on 8899.

Now I went to https://localhost:8443/nifi in my browser and got prompted to request an account. At this point I edited nifi.properties again to turn on regular http access by setting nifi.web.http.port=8080, restarted nifi, went to http://localhost:8080/nifi and approved my account, then removed http access and restarted again, and was able to access over https. I then created a an Output Port called "Test" with a GenerateFlowFile sending data to it.

At this point nifi-https is fully setup.

On nifi-http I configured the following properties in nifi.properties (everything else left as defaults):

nifi.security.keystore=mycert.p12
nifi.security.keystoreType=PKCS12
nifi.security.keystorePasswd=mypassword
nifi.security.truststore=mytruststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=mypassword

That makes nifi-http a regular unsecured instance running on port 8080, but it now has the cert and truststore to make outbound secure connections.

Now I went to http://localhost:8080/nifi and create a Remote Process Group (RPG) with a URL of https://hostname.from.my.cert:8443/nifi. It is import that the hostname in this URL matches the value of "nifi.remote.input.socket.host" from the nifi-https instance.

Now I right-clicked on the RPG and chose Enable Transmission at which point I got a message that an account was requested. This happened because nifi-http is using mycert.p12 to connect to nifi-https, but nifi-https does not have an approved account for mycert.p12.

So I went to nifi-https (https://localhost:8443/nifi) and went to the accounts section and approved the account for mycert.p12 and chose a role of "NiFi'.

We also need to give the mycert.p12 user access to the "Test" Output Port. So on the https instance I stopped "Test", right-clicked and Configure, and from the Access Controls tab started typing the DN from mycert.p12, added that user to the Allowed Users list, hit Apply and started the port again.

Then I went back to nifi-http and right-clicked on the RPG and chose Refresh which caused it to retrieve the available Output Ports from nifi-https. I then connected the "Test" Output Port from the RPG to LogAttribute, started everything and it was able to pull FlowFiles from nifi-https.

View solution in original post

Highlighted

Re: Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

@Bryan Bende What do you mean by "So I went to nifi-https (https://localhost:8443/nifi) and went to the accounts section and approved the account for mycert.p12 and chose a role of "NiFi'." Does that mean you added to the authorized_users file the DN associated with mycert.p12 and added a role of ROLE_Nifi?

Highlighted

Re: Hortonworks HDF( Nifi ) : Site to Site ( https < — > http )

This post is written against Apache NiFi 0.6.1... in that version when you enabled the RPG and it makes a request to the secure instance, it automatically makes an account request for the DN being used by the RPG (mycert.p12). Then someone has to go into the other instance UI and approve that account and give it a role of a NIFI, which behind the scenes updates the authorized_users.xml.

This is a bit different in Apache NiFi 1.0.0... there is no more automatic account requests and the authorization model is very different. Refer this blog post for how to setup the authorizations: http://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site

Don't have an account?
Coming from Hortonworks? Activate your account here