Knox to MIT KDC, socket closed

[#1]

Hi,

Trying to get a Know gateway up and running. Know will not connect to local MIT KDC. On checking with the CLI utils (knoxcli.sh system-user-auth-test & user-auth-test ) we get following error :

/usr/hdp/2.4.2.0-258/knox/bin> ./knoxcli.sh --d system-user-auth-test --cluster default

org.apache.shiro.authc.AuthenticationException: LDAP naming error while attempting to authenticate user. 10.xxx.xxx.x1:88; socket closed
org.apache.shiro.authc.AuthenticationException: LDAP naming error while attempting to authenticate user.
        at org.apache.shiro.realm.ldap.JndiLdapRealm.doGetAuthenticationInfo(JndiLdapRealm.java:303)
        at org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm.doGetAuthenticationInfo(KnoxLdapRealm.java:177)    at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:568)
        at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doSingleRealmAuthentication(ModularRealmAuthenticator.java:180)
        at org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:267) at org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198)
        at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106)
        at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:270)
        at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256)
        at org.apache.hadoop.gateway.util.KnoxCLI$LDAPCommand.authenticateUser(KnoxCLI.java:1037)
        at org.apache.hadoop.gateway.util.KnoxCLI$LDAPCommand.testSysBind(KnoxCLI.java:1139)
        at org.apache.hadoop.gateway.util.KnoxCLI$LDAPSysBindCommand.execute(KnoxCLI.java:1446)
        at org.apache.hadoop.gateway.util.KnoxCLI.run(KnoxCLI.java:138)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.gateway.util.KnoxCLI.main(KnoxCLI.java:1643)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
        at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
        at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
        at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
        at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: javax.naming.ServiceUnavailableException: 10.xxx.xxx.x1:88; socket closed
        at com.sun.jndi.ldap.Connection.readReply(Connection.java:454)
        at com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:365)
        at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:214)
        at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2788)
        at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319)
        at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192)
        at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210)
        at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153)
        at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83)
        at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
        at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313)
        at javax.naming.InitialContext.init(InitialContext.java:244)
        at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:154)
        at org.apache.shiro.realm.ldap.JndiLdapContextFactory.createLdapContext(JndiLdapContextFactory.java:508) at org.apache.shiro.realm.ldap.JndiLdapContextFactory.getLdapContext(JndiLdapContextFactory.java:495)    at org.apache.shiro.realm.ldap.JndiLdapRealm.queryForAuthenticationInfo(JndiLdapRealm.java:375)
        at org.apache.shiro.realm.ldap.JndiLdapRealm.doGetAuthenticationInfo(JndiLdapRealm.java:295)
        ... 23 more
Unable to successfully bind to LDAP server with topology credentials. Are your parameters correct?

Socket is definitely open and reachable, since we can get to it with netcat and telnet. Knox system user used can login to kadmin without problems.

Any idea's ?

topology is below:

<topology>
    <gateway>
        <provider>
            <role>authentication</role>
            <name>ShiroProvider</name>
            <enabled>true</enabled>
            <param>
                <name>sessionTimeout</name>
                <value>30</value>
            </param>
            <param>
                <name>main.ldapRealm</name>
                <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
            </param>
            <!-- changes for AD/user sync -->
            <param>
                <name>main.ldapContextFactory</name>
                <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
            </param>
            <!-- main.ldapRealm.contextFactory needs to be placed before other main.ldapRealm.contextFactory* entries  -->
            <param>
                <name>main.ldapRealm.contextFactory</name>
                <value>$ldapContextFactory</value>
            </param>
            <!-- AD url -->
            <param>
                <name>main.ldapRealm.contextFactory.url</name>
                <value>ldap://xxxxxxxxxxxxx:88</value>
            </param>
            <!-- system user -->
            <param>
                <name>main.ldapRealm.contextFactory.systemUsername</name>
                <value>CN=admin,DC=HADOOP,DC=COM</value>
            </param>
            <!-- pass in the password using the alias created earlier -->
            <param>
                <name>main.ldapRealm.contextFactory.systemPassword</name>
                <value>#####</value>
            </param>
      <!--      <param>
                <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
                <value>kerberos</value>
            </param> -->
            <param>
                <name>urls./**</name>
                <value>authcBasic</value>
            </param>
            <!--  AD groups of users to allow -->
            <param>
                <name>main.ldapRealm.searchBase</name>
                <value>DC=HADOOP,DC=COM</value>
            </param>
            <param>
                <name>main.ldapRealm.userObjectClass</name>
                <value>person</value>
            </param>
            <param>
                <name>main.ldapRealm.userSearchAttributeName</name>
                <value>sAMAccountName</value>
            </param>
            <!-- changes needed for group sync-->
            <param>
                <name>main.ldapRealm.authorizationEnabled</name>
                <value>true</value>
            </param>
            <param>
                <name>main.ldapRealm.groupSearchBase</name>
                <value>DC=HADOOP,DC=COM</value>
            </param>
            <param>
                <name>main.ldapRealm.groupObjectClass</name>
                <value>group</value>
            </param>
            <param>
                <name>main.ldapRealm.groupIdAttribute</name>
                <value>cn</value>
            </param>
        </provider>
        <provider>
            <role>identity-assertion</role>
            <name>Default</name>
            <enabled>true</enabled>
        </provider>
        <provider>
            <role>authorization</role>
            <name>XASecurePDPKnox</name>
            <enabled>true</enabled>
        </provider>
    </gateway>
    <service>
        <role>NAMENODE</role>
        <url>hdfs://{{namenode_host}}:{{namenode_rpc_port}}</url>
    </service>
    <service>
        <role>JOBTRACKER</role>
        <url>rpc://{{rm_host}}:{{jt_rpc_port}}</url>
    </service>
    <service>
        <role>WEBHDFS</role>
        <url>http://{{namenode_host}}:{{namenode_http_port}}/webhdfs</url>
    </service>
    <service>
        <role>WEBHCAT</role>
        <url>http://{{webhcat_server_host}}:{{templeton_port}}/templeton</url>
    </service>
    <service>
        <role>OOZIE</role>
        <url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url>
    </service>
    <service>
        <role>WEBHBASE</role>
        <url>http://{{hbase_master_host}}:{{hbase_master_port}}</url>
    </service>
    <service>
        <role>HIVE</role>
        <url>http://{{hive_server_host}}:{{hive_http_port}}/{{hive_http_path}}</url>
    </service>
    <service>
        <role>RESOURCEMANAGER</role>
        <url>http://{{rm_host}}:{{rm_port}}/ws</url>
    </service>
<service>
    <role>YARNUI</role>
    <url>http://{{rm_host}}:{{rm_port}}</url>
  </service>
</topology>

[#2]

@Jasper

The first thing I see that is probably incorrect is your UTL for the main.ldapRealm.contextFactory.url. You specify port 88 of the URL. LDAP listens on port 389 by default (636 for LDAPS).

Change:

 <value>ldap://xxxxxxxxxxxxx:88</value>

to

 <value>ldap://xxxxxxxxxxxxx:389</value>

That should get you along the way.

[#3]

@emaxwell we checked the processes and ports listening on the KDC host, but this surely seemed to be the KDC. 389/636 were not there. I think these can be altered to something non default. But will check it again.

[#4]

@emaxwell

Some extra info about the environment:

We are trying to go directly against the KDC here. :88 is the default port for that. OpenLdap is not available. So can Knox be configured to do just that, without OpenLDAP as middleware?

[#5 · Accepted solution]

Shiro Provider is used to authenticate users, when they enter user-name/password against an LDAP store. See here for details. "main.ldapRealm.contextFactory.url" must be $protocol :// $ldaphost : $port, where $protocol is ldap or ldaps pointing to a node which provides LDAP service like OpenLDAP, AD, FreeIPA, or the test LDAP server bundled with Knox, running on port 33389. KDC doesn't provide LDAP service.

[#6]

Turns out Knox can not be set up easily when you have installed just the local MIT KDC for kerberizing the cluster as per this doc . Or you would have to at least install additional services on top like @Predrag Minovic mentioned.

That is the very reason why Knox ships with the example demo LDAP that you can start at the Ambari Knox service config Service Actions > Start Demo LDAP.