Support Questions

Find answers, ask questions, and share your expertise

How to connect Site to Site from HTTP to HTTPS.??

Super Collaborator

Hi,

We have a HDF server inside our company's network and we just secured our remote server (one server only) on our HADOOP cluster with LDAP authentication (outside of our network , managed by rackspace).

How do i connect to a remote process using Site 2 Site..? Earlier i have my putHDFS process on remote server and i was sending files using Site 2 Site from local server. Since both are HTTP it was working fine then.

But Now i am not able to connect to the remote port using because of secure connection. (HTTPs\LDAP etc).

What is the solution.? Do i have to go HTTPS on the local server too with the same LDAP settings..??

Thank you.

Regards,

Sai

1 ACCEPTED SOLUTION

@Saikrishna Tarapareddy @Matt

A unsecured NiFi instance can interact with a secured NiFi instancy, if it's configured with its keystore and truststore, and also the remote NiFi has a proper user and policies configured with it.

I have a working example on Github, nifi-integration-test, that confirms inter connectivity among unsecured/secured/standalone/clustered NiFi instances, in which you can find example nifi.properties, keystore and truststore. Here are important properties to configure:

  • Unsecured NiFi (local instance in your use case)
    • nifi.web.http.port=8080
    • nifi.web.https.port= (blank)
    • nifi.security.keystore=(configure keystore and truststore)
    • e.g. nifi-sp.properties
  • Secured NiFi (remote instance in your use case)
    • nifi.web.http.port=(blank)
    • nifi.web.https.port= 8443
    • nifi.security.keystore=(configure keystore and truststore)
    • nifi.remote.input.secure=true
    • e.g. nifi-ss.properties

Hope this helps,

Koji

View solution in original post

13 REPLIES 13

Master Guru

@Saikrishna Tarapareddy

S2S does not use LDAP for server authentication. S2S uses the keystore and truststore provided in the nifi.properties file to establish a secured mutual authenticated connection between two secured NiFi instances/clusters. The destination NiFi dictate whether the S2S connection will be secure or not. If you have secured your destination NiFi, then only a source NiFi (one with the RPG) that has been configured with its own server keystore and truststore will be able to connect.

Since S2S relies on certificates for mutual authentication. The user authentication you choose to use can be different on each NiFi installation. LDAP on one, certs on another, etc...

Thanks,

Matt

Super Collaborator

So this means i cannot access a HTTPS Nifi instance from a HTTP using RPG?

Is there a way where i can type in username and password of the HTTPS nifi and get thru from HTTP nifi.?

just trying to see if there is an easier way without going my local instance to HTTPS.

But if i have to go HTTPs on the local instance , do i need same keystore and truststore of the server.?

Thanks,

Sai

@Saikrishna Tarapareddy @Matt

A unsecured NiFi instance can interact with a secured NiFi instancy, if it's configured with its keystore and truststore, and also the remote NiFi has a proper user and policies configured with it.

I have a working example on Github, nifi-integration-test, that confirms inter connectivity among unsecured/secured/standalone/clustered NiFi instances, in which you can find example nifi.properties, keystore and truststore. Here are important properties to configure:

  • Unsecured NiFi (local instance in your use case)
    • nifi.web.http.port=8080
    • nifi.web.https.port= (blank)
    • nifi.security.keystore=(configure keystore and truststore)
    • e.g. nifi-sp.properties
  • Secured NiFi (remote instance in your use case)
    • nifi.web.http.port=(blank)
    • nifi.web.https.port= 8443
    • nifi.security.keystore=(configure keystore and truststore)
    • nifi.remote.input.secure=true
    • e.g. nifi-ss.properties

Hope this helps,

Koji

Super Collaborator

@kkawamura

Hi ,

1. it needs the same keystore and truststore on both servers.?

2. do I have to grant access to input port on the remote server.? if so for which user I should give as I am running local nifi as default user.

regards,

sai

@Saikrishna Tarapareddy

Hi,

1. No, different keystore and truststore can be used.

2. Yes, you have to grant the user access to 'retrieve site-to-site' and 'receive data via site-to-site'. The remote NiFi will authenticate the local NiFi using local NiFi's client cert DN. You might have to map the DN to a LDAP user if LDAP is used to authorization. You can find following settings to map user identities among different authentication mechanisms:

# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.security.identity.mapping.value.dn=$1@$2

I haven't tried LDAP into the mix, so when I have time, I'll try to confirm it works as expected.

Regards,

Koji

Master Guru

@Saikrishna Tarapareddy

S2S will not use LDAP for authentication. It uses the DN from the client side cert you created/obtained. If the Identity mapping properties @Koji Kamimura mentioned above are configured on the receiving side NiFi, they will be applied against that client side certificate DN that is presented. The resulting mapped value is what will need to be authorized to access S2S details and any input ports you wish to post to. If the identity mapping stuff is not configured, the full DN will need to authorized.

Thanks,

Matt

@Matt Thanks for the clarification!

@Saikrishna Tarapareddy I tried setup a secured NiFi instance configured with LDAP and access it from a unsecured NiFi instance. It worked as expected. I wrote up a summary in this blog.

http://ijokarumawak.github.io/nifi/2016/11/15/nifi-auth/

Please let us know if you need further assistance.

Thanks,

Koji

Super Collaborator

@kkawamura @Matt

Based on your recommendations , Here is what I did . But I still cant see the remote input port.

I used these commands to create keystore and truststore on my local HDF server which is HTTP , copied on to my Config folder..

9460-s2s1.png

and updated the config file as

nifi.security.keystore=./conf/localhost.keystore.jks nifi.security.keystoreType=JKS nifi.security.keystorePasswd=**** nifi.security.keyPasswd=***** nifi.security.truststore=./conf/localhost.truststore.jks nifi.security.truststoreType=JKS nifi.security.truststorePasswd=*****

and on my remote instance which is HTTPS with LDAP..i logged in as admin and created a user matching DN of the client..like below..

9471-s2s2.png

and granted access to the retrieve site-to-smote policy and input port..

9472-s2s3.png

9473-s2s4.png

and my flow on the remote server is

9474-iwpri.png

when i drag RPG and point that to remote server..i am getting this error below.

3:50:10 CST WARNING 697e84b3-0158-1000-bd67-1e28e1517603

Unable to connect to https://localhost:11090/nifi due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Regards,

Sai

Hi @Saikrishna Tarapareddy

Did you add the remote NiFi's cert or its CA cert to local NiFi's truststore? You also need to add the local NiFi cert or its CA cert to the remote NiFi's truststore.

The error message usually indicates that the cert is not trusted because it was unable to find valid certification path.

Please use keytool command looks like below, to add certs into a truststore:

keytool -importcert -file certificate.cer -keystore keystore.jks -alias "Alias"

Thanks,

Koji

Super Collaborator

@kkawamura @Matt

i have exported remote server certificate to local nifi and local nifi certificate to server. but now i am getting a forbidden error.

i checked and granted all the permissions needed for user "CN=nifistos, OU=nestle" on the https system.

2016-11-22 15:48:59,285 INFO [NiFi Web Server-36] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=nifistos, OU=nestle) GET https://myremoteserver:10090/nifi-api/site-to-site (source ip: 172.16.2.4)

2016-11-22 15:48:59,285 INFO [NiFi Web Server-36] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=nifistos, OU=nestle

2016-11-22 15:48:59,286 INFO [NiFi Web Server-36] o.a.n.w.a.c.AccessDeniedExceptionMapper CN=nifistos, OU=nestle does not have permission to access the requested resource. Returning Forbidden response.

anything else that i am missing..??

Hi @Saikrishna Tarapareddy

Please double check the 'CN=nifistos, OU=nestle' user has right policy settings to use Site-to-Site. The user needs

  • 'retrieve site-to-site details' in global policies (top-right menu)
  • 'receive data via site-to-site' policy of the Input Port (select the Input Port and click the key icon on the operation palette on the left)

Once the user got authenticated, the all authorization checks the user has to pass are above two policies.

Super Collaborator

@kkawamura@Matt

found the issue , my Server user has needs a space between CN and OU.

thanks for the help.

@Saikrishna Tarapareddy

I hadn't read this comment when I wrote a reply few seconds ago. Glad to hear that it worked!

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.