How to connect Site to Site from HTTP to HTTPS.??

[#1]

Hi,

We have a HDF server inside our company's network and we just secured our remote server (one server only) on our HADOOP cluster with LDAP authentication (outside of our network , managed by rackspace).

How do i connect to a remote process using Site 2 Site..? Earlier i have my putHDFS process on remote server and i was sending files using Site 2 Site from local server. Since both are HTTP it was working fine then.

But Now i am not able to connect to the remote port using because of secure connection. (HTTPs\LDAP etc).

What is the solution.? Do i have to go HTTPS on the local server too with the same LDAP settings..??

Thank you.

Regards,

Sai

[#2]

@Saikrishna Tarapareddy

S2S does not use LDAP for server authentication. S2S uses the keystore and truststore provided in the nifi.properties file to establish a secured mutual authenticated connection between two secured NiFi instances/clusters. The destination NiFi dictate whether the S2S connection will be secure or not. If you have secured your destination NiFi, then only a source NiFi (one with the RPG) that has been configured with its own server keystore and truststore will be able to connect.

Since S2S relies on certificates for mutual authentication. The user authentication you choose to use can be different on each NiFi installation. LDAP on one, certs on another, etc...

Thanks,

Matt

[#3]

So this means i cannot access a HTTPS Nifi instance from a HTTP using RPG?

Is there a way where i can type in username and password of the HTTPS nifi and get thru from HTTP nifi.?

just trying to see if there is an easier way without going my local instance to HTTPS.

But if i have to go HTTPs on the local instance , do i need same keystore and truststore of the server.?

Thanks,

Sai

[#4 · Accepted solution]

@Saikrishna Tarapareddy @Matt

A unsecured NiFi instance can interact with a secured NiFi instancy, if it's configured with its keystore and truststore, and also the remote NiFi has a proper user and policies configured with it.

I have a working example on Github, nifi-integration-test, that confirms inter connectivity among unsecured/secured/standalone/clustered NiFi instances, in which you can find example nifi.properties, keystore and truststore. Here are important properties to configure:

• Unsecured NiFi (local instance in your use case)
  • nifi.web.http.port=8080
  • nifi.web.https.port= (blank)
  • nifi.security.keystore=(configure keystore and truststore)
  • e.g. nifi-sp.properties
• Secured NiFi (remote instance in your use case)
  • nifi.web.http.port=(blank)
  • nifi.web.https.port= 8443
  • nifi.security.keystore=(configure keystore and truststore)
  • nifi.remote.input.secure=true
  • e.g. nifi-ss.properties

Hope this helps,

Koji

[#5]

@kkawamura

Hi ,

1. it needs the same keystore and truststore on both servers.?

2. do I have to grant access to input port on the remote server.? if so for which user I should give as I am running local nifi as default user.

regards,

sai

[#6]

@Saikrishna Tarapareddy

Hi,

1. No, different keystore and truststore can be used.

2. Yes, you have to grant the user access to 'retrieve site-to-site' and 'receive data via site-to-site'. The remote NiFi will authenticate the local NiFi using local NiFi's client cert DN. You might have to map the DN to a LDAP user if LDAP is used to authorization. You can find following settings to map user identities among different authentication mechanisms:

# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.security.identity.mapping.value.dn=$1@$2

I haven't tried LDAP into the mix, so when I have time, I'll try to confirm it works as expected.

Regards,

Koji

[#7]

@Saikrishna Tarapareddy

S2S will not use LDAP for authentication. It uses the DN from the client side cert you created/obtained. If the Identity mapping properties @Koji Kamimura mentioned above are configured on the receiving side NiFi, they will be applied against that client side certificate DN that is presented. The resulting mapped value is what will need to be authorized to access S2S details and any input ports you wish to post to. If the identity mapping stuff is not configured, the full DN will need to authorized.

Thanks,

Matt

[#8]

@Matt Thanks for the clarification!

@Saikrishna Tarapareddy I tried setup a secured NiFi instance configured with LDAP and access it from a unsecured NiFi instance. It worked as expected. I wrote up a summary in this blog.

http://ijokarumawak.github.io/nifi/2016/11/15/nifi-auth/

Please let us know if you need further assistance.

Thanks,

Koji

[#9]

@kkawamura @Matt

Based on your recommendations , Here is what I did . But I still cant see the remote input port.

I used these commands to create keystore and truststore on my local HDF server which is HTTP , copied on to my Config folder..

and updated the config file as

nifi.security.keystore=./conf/localhost.keystore.jks nifi.security.keystoreType=JKS nifi.security.keystorePasswd=**** nifi.security.keyPasswd=***** nifi.security.truststore=./conf/localhost.truststore.jks nifi.security.truststoreType=JKS nifi.security.truststorePasswd=*****

and on my remote instance which is HTTPS with LDAP..i logged in as admin and created a user matching DN of the client..like below..

and granted access to the retrieve site-to-smote policy and input port..

and my flow on the remote server is

when i drag RPG and point that to remote server..i am getting this error below.

3:50:10 CST WARNING 697e84b3-0158-1000-bd67-1e28e1517603

Unable to connect to https://localhost:11090/nifi due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Regards,

Sai

[#10]

Hi @Saikrishna Tarapareddy

Did you add the remote NiFi's cert or its CA cert to local NiFi's truststore? You also need to add the local NiFi cert or its CA cert to the remote NiFi's truststore.

The error message usually indicates that the cert is not trusted because it was unable to find valid certification path.

Please use keytool command looks like below, to add certs into a truststore:

keytool -importcert -file certificate.cer -keystore keystore.jks -alias "Alias"

Thanks,

Koji