Nifi with Ldap Authenitcation

[#1]

Hi All,

We are facing some issue with the ldap integratin with Nifi standalone.

My issue is if i enabled Ldap i didnt get the login page and i didnt see any errors related to ldap in log.

Few questions I have ;

1) if i used ldap aunthentication do i need to generate ssl for the same user to access/accept from browser?

2) Suppose if we have 30 Ldap users want to access Nifi for all the 30 users do i need to generate SSL cert ?

Please find my below config.

This is my login-identity-providers.xml

<provider>
        <identifier>ldap-provider</identifier>
        <class>org.apache.nifi.ldap.LdapProvider</class>
        <property name="Authentication Strategy">SIMPLE</property>
        <property name="Manager DN">sAMAccountName=admin,OU=Users,OU=Accounts,OU=ITSC,DC=zone1,DC=XXX,DC=net</property>
        <property name="Manager Password">Password#</property>
        <property name="TLS - Keystore">/opt/install/HDF-2.0.1.0/nifi/conf/test/server_keystore.jks</property>
        <property name="TLS - Keystore Password">SuperSecret</property>
        <property name="TLS - Keystore Type">JKS</property>
        <property name="TLS - Truststore">/opt/install/HDF-2.0.1.0/nifi/conf/test/server_truststore.jks</property>
        <property name="TLS - Truststore Password">SuperSecret</property>
        <property name="TLS - Truststore Type">JKS</property>
        <property name="TLS - Client Auth"></property>
        <property name="TLS - Protocol">TLS</property>
        <property name="TLS - Shutdown Gracefully"></property>
        <property name="Referral Strategy">FOLLOW</property>
        <property name="Connect Timeout">10 secs</property>
        <property name="Read Timeout">10 secs</property>
        <property name="Url">ldap://ldap.zone1.XXX.net:389</property>
        <property name="User Search Base">OU=Users,OU=Accounts,OU=ITSC,DC=zone1,DC=XXX,DC=net</property>
        <property name="User Search Filter">sAMAccountName={0}</property>
        <property name="Authentication Expiration">12 hours</property>
        </provider>

authorizers.xml

<authorizer>
        <identifier>file-provider</identifier>
        <class>org.apache.nifi.authorization.FileAuthorizer</class>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial Admin Identity"></property>
        <property name="Legacy Authorized Users File"></property>
        <!-- Provide the identity (typically a DN) of each node when clustered, see above description of Node Identity.
        <property name="Node Identity 1"></property>
        <property name="Node Identity 2"></property>
        -->
    </authorizer>

And Nifi.properties.

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Core Properties #
nifi.version=1.0.0.2.0.1.0-12
nifi.flow.configuration.file=./conf/flow.xml.gz
nifi.flow.configuration.archive.enabled=true
nifi.flow.configuration.archive.dir=./conf/archive/
nifi.flow.configuration.archive.max.time=30 days
nifi.flow.configuration.archive.max.storage=500 MB
nifi.flowcontroller.autoResumeState=true
nifi.flowcontroller.graceful.shutdown.period=10 sec
nifi.flowservice.writedelay.interval=500 ms
nifi.administrative.yield.duration=30 sec
# If a component has no work to do (is "bored"), how long should we wait before checking again for work?
nifi.bored.yield.duration=10 millis
nifi.authorizer.configuration.file=./conf/authorizers.xml
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.templates.directory=./conf/templates
nifi.ui.banner.text=
nifi.ui.autorefresh.interval=30 sec
nifi.nar.library.directory=./lib
nifi.nar.working.directory=./work/nar/
nifi.documentation.working.directory=./work/docs/components
####################
# State Management #
####################
nifi.state.management.configuration.file=./conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
# The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster.
nifi.state.management.provider.cluster=zk-provider
# Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server
nifi.state.management.embedded.zookeeper.start=false
# Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
# H2 Settings
nifi.database.directory=./database_repository
nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.directory=./flowfile_repository
nifi.flowfile.repository.partitions=256
nifi.flowfile.repository.checkpoint.interval=2 mins
nifi.flowfile.repository.always.sync=false
nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000
nifi.swap.in.period=5 sec
nifi.swap.in.threads=1
nifi.swap.out.period=5 sec
nifi.swap.out.threads=4
# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=10 MB
nifi.content.claim.max.flow.files=100
nifi.content.repository.directory.default=./content_repository
nifi.content.repository.archive.max.retention.period=12 hours
nifi.content.repository.archive.max.usage.percentage=50%
nifi.content.repository.archive.enabled=true
nifi.content.repository.always.sync=false
nifi.content.viewer.url=/nifi-content-viewer/
# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository
# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=./provenance_repository
nifi.provenance.repository.max.storage.time=24 hours
nifi.provenance.repository.max.storage.size=1 GB
nifi.provenance.repository.rollover.time=30 secs
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=1
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
nifi.provenance.repository.journal.count=16
# Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable.  Some examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
# but should provide better performance
nifi.provenance.repository.index.shard.size=500 MB
# Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
# Volatile Provenance Respository Properties
nifi.provenance.repository.buffer.size=100000
# Component Status Repository
nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository
nifi.components.status.repository.buffer.size=1440
nifi.components.status.snapshot.frequency=1 min
# Site to Site properties
nifi.remote.input.host=
nifi.remote.input.secure=false
nifi.remote.input.socket.port=
nifi.remote.input.https.enabled=true
nifi.remote.input.https.transaction.ttl=30 sec
# web properties #
nifi.web.war.directory=./lib
#nifi.web.http.host=
#nifi.web.http.port=
nifi.web.https.host=
nifi.web.https.port=7443
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
# security properties #
nifi.sensitive.props.key=
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=
nifi.security.keystore=/opt/install/HDF-2.0.1.0/nifi/conf/test/server_keystore.jks
nifi.security.keystoreType=JKS
nifi.security.keystorePasswd=SuperSecret
nifi.security.keyPasswd=SuperSecret
nifi.security.truststore=/opt/install/HDF-2.0.1.0/nifi/conf/test/server_truststore.jks
nifi.security.truststoreType=JKS
nifi.security.truststorePasswd=SuperSecret
nifi.security.needClientAuth=
nifi.security.user.authorizer=file-provider
nifi.security.user.login.identity.provider=ldap-provider
nifi.security.ocsp.responder.url=
nifi.security.ocsp.responder.certificate=
# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.security.identity.mapping.value.dn=$1@$2
# nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
# nifi.security.identity.mapping.value.kerb=$1@$2
# cluster common properties (all nodes must have same values) #
nifi.cluster.protocol.heartbeat.interval=5 sec
nifi.cluster.protocol.is.secure=false
# cluster node properties (only configure for cluster nodes) #
nifi.cluster.is.node=false
nifi.cluster.node.address=
nifi.cluster.node.protocol.port=
nifi.cluster.node.protocol.threads=10
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=5 sec
nifi.cluster.node.read.timeout=5 sec
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=5 mins
nifi.cluster.flow.election.max.candidates=
# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=
nifi.zookeeper.connect.timeout=3 secs
nifi.zookeeper.session.timeout=3 secs
nifi.zookeeper.root.node=/nifi
# kerberos #
#nifi.kerberos.krb5.file=
# kerberos service principle #
#nifi.kerberos.service.principal=
#nifi.kerberos.service.keytab.location=
# kerberos spnego principle #
#nifi.kerberos.spnego.principal=
#nifi.kerberos.spnego.keytab.location=
#nifi.kerberos.spnego.authentication.expiration=12 hours
# external properties files for variable registry
# supports a comma delimited list of file locations
nifi.variable.registry.properties=

[#2]

@Bryan Bende Can you guide on this.

[#3]

Matt's answer below is correct....

If you are doing LDAP authentication then none of your end users need certificates, only the NiFi nodes themselves do, NiFi still needs to be running on https.

In you config above your Initial Admin is blank, you always want to provide a value for this... If you are using certificates authentication then it is the DN of your cert, if you are using LDAP then it is the DN of an LDAP users, and if you are using Kerberos then it is the principal of your Kerberos user.

[#4]

@Bryan Bende "only the NiFi nodes themselves do, NiFi still needs to be running on https."

It means only SSL cert for the Node? for example if my Nifi is running on the domain nifi.test.com for this domain do i need to generate SSL? "Self signed cert will work" for testing in the ldap environment.

And evertime even if i gave LDAP users DN given in the "Initial Admin" still its comparing With SSL cert value and its saying "Unable to perform the desired action due to insufficient permissions. Contact the system administrator."

Can you guide on this.

[#5 · Accepted solution]

@Anishkumar Valsalam

NiFi supports three types of user authentication strategies:

1. SSL Certificates - This is the default authentication strategy and is always on. NiFi will always accept a user supplied SSL certificate in the connection before trying any other configured authentication strategy. If a SSL certificate is not presented in the connection, NiFi will close the connection unless another authentication strategy has been configured. Those other strategy must be configured and include:

2. LDAP - NiFi can be configured to fall back on this strategy which can be configured to connect to either LDAP or LDAPS. If your user use LDAP, they do not need to have user SSL certificates issued to them. If they do have SSL certificates loaded in their browser, NiFi will accept them and will not even try using LDAP to authenticate that user.

3. Kerberos - NiFi can be configured to fall back on this strategy. If they do have SSL certificates loaded in their browser, NiFi will accept them and will not even try using Kerberos to authenticate that user.

Note: Alternate authentication strategies are configured in NiFi's login-identity-providers.xml file. NiFi SSL certificate authentication is the default (first strategy that is always attempted) and cannot be disabled. NiFi will only try one of the other authentication strategies (if configured) if SSL certificates are not presented for the user.

Note: Only one alternative authentication Strategy can be configured. While SSL is always enabled, you can only add either LDAP or Kerberos, but not both.

Note: Authentication is only step one or a two step process that needs to be successful before a user can successfully access NiFi. NiFi authorization occurs after successful authentication. User who can successfully authenticate via any one of the above authentication strategies still need to be authorized within NiFi for access to specific NiFi resources and components. Authorization of users requires adding those users to the configured authorizer and granted those added users access to whichever access policies are needed by that user.

Thanks,

Matt

[#6]

@Anishkumar Valsalam

Trying to keep one answer going here rather then starting a new answer to add details. 🙂

What do you see in your nifi-user.log?

I suspect you see on a line that says authentication success for user XYZ.

then another line that says user XYZ has insufficient permissions.

NiFi uses the DN returned by LDAP to perform its authorization lookup.

You have to make sure that user "XYZ" was added as a user in your NiFi. That user must be added exactly as you see it. It is case and sensitive and should not have any quotes around it. Be mindful of any spaces between DN elements or unexpected leading or trailing spaces.

At a minimum, that new user must be granted the “view the user interface” access policy or they will not be able to load the NiFi UI. They will need additional policies granted to them to do other things once they access the UI, but that is for you to decide what you want to grant each user access to do.

Matt

[#7]

@Matt Sure 🙂

Please find the Log.initial message i am seeing successs but after that iam seeing "AccessDeniedException: Unknown user with identity" exception.

My question is for all users i need to add all the DN details.

like below

cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net

Using username(o.1548691)nifi will not do automatic lookup?

2017-01-25 02:05:30,069 INFO [NiFi Web Server-46] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net
2017-01-25 02:05:30,069 DEBUG [NiFi Web Server-46] o.a.n.w.s.NiFiAuthenticationFilter Checking secure context token: cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net
2017-01-25 02:05:30,069 DEBUG [NiFi Web Server-46] o.a.n.w.s.a.NiFiAnonymousUserFilter SecurityContextHolder not populated with anonymous token, as it already contained: 'cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net'
2017-01-25 02:05:30,070 INFO [NiFi Web Server-46] o.a.n.w.a.c.AccessDeniedExceptionMapper cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net does not have permission to access the requested resource. Returning Forbidden response.
2017-01-25 02:05:30,071 DEBUG [NiFi Web Server-46] o.a.n.w.a.c.AccessDeniedExceptionMapper
org.apache.nifi.authorization.AccessDeniedException: Unknown user with identity 'cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net'.
        at org.apache.nifi.web.api.FlowResource.authorizeFlow(FlowResource.java:226) ~[classes/:na]
        at org.apache.nifi.web.api.FlowResource.getCurrentUser(FlowResource.java:312) ~[classes/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) [jersey-servlet-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) [jersey-servlet-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) [jersey-servlet-1.19.jar:1.19]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1689) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:51) [jetty-servlets-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) [classes/:na]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316) [spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]

[#8]

@Anishkumar Valsalam

You need to add this as a new user in NiFi:

cn=o.1548691,ou=Users,ou=Accounts,ou=ITSC,dc=zone1,dc=testdev,dc=net

Click "Users" to open the users and groups NiFi UI. The "individual" "identity" string must match EXACTLY with what you saw in the nifi-user.log. It is case sensitive and make sure you have not copied in the single quotes or any leading or trailing spaces.

Click on "Policies" to access the below UI and grant this newly added user a minimum of "view the user interface":

Thanks,

Matt

[#9]

@Matt

Thanks Much 🙂 It works like charm. and thanks for your patience too 🙂

[#10]

Hi Matt, I am configuration User Authentication with LDAP and present the following error:

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProvider': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider 'ldap-provider' could not be found. at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:254) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060) ~[spring-context-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:90) ~[na:na] ... 28 common frames omitted Caused by: java.lang.Exception: The specified login identity provider 'ldap-provider' could not be found. at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.getObject(LoginIdentityProviderFactoryBean.java:121) ~[nifi-web-security-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]

Please help me.

Regards,

Julio