could not load AD group members

[#1]

hi i'm trying to integrate AD with hue on cloudera manager. and the thing is hue could not retrieve AD group members when sync users and groups.  my configuration is like below:

Authentication Backend = desktop.auth.backend.LdapBackend

ldap_url = ldap://example.ap.example.net
nt_domain = ap.example.net
base_dn = DC=ap, DC=example, DC=net
bind_dn = admin
bind_password = password
user_filter = objectclass=*
user_name_attr = sAMAccountName
group_filter = objectclass=*
group_name_attr = cn

group_member_attr = member

could anyone help to figure out the issue?

thanks!

[#2]

The bind_dn should be the full DN path for the bind user.

i.e. bind_dn="CN=ServiceAccount,DC=mycompany,DC=com"

[#3]

Hi, jjiang. You are missing the port on the LDAP URL. Also, the format looks wrong.  

It should be  ldap://<ldap_server>:389 (or  ldaps://<ldap_server>:636). 

Your Bind DN is fine. It should only be the full path when binding with Username Pattern. You are binding with NT Domain so the Bind DN should only be the username. 

Detailed docs and videos are coming in the Hue Guide with the next release.

[#4]

yeah, i'm using direct bind. and i set the port as well, it doesnt work. there are actually no communication  error between AD server and hue even if i didnt set the port. i can logon hue with AD user. 

the only thing is i cant sync the AD groups and membership. dont know why, and could not see any error log.

[#5]

Hm, strange. In theory, this should work:

To import and synchronize one group (and its multiple users):

1. Log on to the Hue UI as a superuser.
2. Go to User Admin > Groups.
3. Click Add/Sync LDAP group.
4. Check Create home directories, and click Sync.

I'm guessing you did this?

[#6]

Also, to automatically synchronize users at the Hue login:

1. Log on to Cloudera Manager and click Hue.
2. Click the Configuration tab and filter by scope=Service-wide and category=Advanced.
3. Configure Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini:

[desktop]
[[ldap]]
sync_groups_on_login=true

4. Click Save Changes and Restart Hue.

[#7]

yes, i already synced the group. and when i set 

sync_groups_on_login=true

 it comes with below when i logon hue:

Traceback (most recent call last):
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1215, in communicate
    req.respond()
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 576, in respond
    self._respond()
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 588, in _respond
    response = self.wsgi_app(self.environ, self.start_response)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/handlers/wsgi.py", line 206, in __call__
    response = self.get_response(request)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/handlers/base.py", line 194, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/handlers/base.py", line 236, in handle_uncaught_exception
    return callback(request, **param_dict)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/views.py", line 336, in serve_500_error
    return render("500.mako", request, {'traceback': traceback.extract_tb(exc_info[2])})
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/django_util.py", line 227, in render
    **kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/django_util.py", line 148, in _render_to_response
    return django_mako.render_to_response(template, *args, **kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/django_mako.py", line 125, in render_to_response
    return HttpResponse(render_to_string(template_name, data_dictionary), **kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/django_mako.py", line 114, in render_to_string_normal
    result = template.render(**data_dict)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/runtime.py", line 786, in _render
    **_kwargs_for_callable(callable_, data))
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/runtime.py", line 818, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/runtime.py", line 844, in _exec_template
    callable_(context, *args, **kwargs)
  File "/tmp/tmpasVufF/desktop/500.mako.py", line 111, in render_body
    __M_writer(unicode( commonfooter(request, messages) ))
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/views.py", line 447, in commonfooter
    'tours_and_tutorials': hue_settings.tours_and_tutorials
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/django_mako.py", line 114, in render_to_string_normal
    result = template.render(**data_dict)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/runtime.py", line 786, in _render
    **_kwargs_for_callable(callable_, data))
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/runtime.py", line 818, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Mako-0.8.1-py2.6.egg/mako/runtime.py", line 844, in _exec_template
    callable_(context, *args, **kwargs)
  File "/tmp/tmpasVufF/desktop/common_footer.mako.py", line 43, in render_body
    __M_writer(unicode( smart_unicode(login_modal(request).content) ))
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/views.py", line 428, in login_modal
    return desktop.auth.views.dt_login(request, True)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/django_axes-1.5.0-py2.6.egg/axes/decorators.py", line 304, in decorated_login
    response = func(request, *args, **kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/auth/views.py", line 115, in dt_login
    if auth_form.is_valid():
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/forms/forms.py", line 129, in is_valid
    return self.is_bound and not bool(self.errors)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/forms/forms.py", line 121, in errors
    self.full_clean()
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/forms/forms.py", line 274, in full_clean
    self._clean_form()
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/forms/forms.py", line 300, in _clean_form
    self.cleaned_data = self.clean()
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/auth/forms.py", line 82, in clean
    return self.authenticate()
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/auth/forms.py", line 102, in authenticate
    server=server)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/contrib/auth/__init__.py", line 49, in authenticate
    user = backend.authenticate(**credentials)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/lib/metrics/registry.py", line 388, in wrapper
    return fn(*args, **kwargs)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/auth/backend.py", line 474, in authenticate
    self.import_groups(server, user)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/desktop/core/src/desktop/auth/backend.py", line 485, in import_groups
    import_ldap_users(connection, user.username, sync_groups=True, import_by_dn=False, server=server)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/apps/useradmin/src/useradmin/views.py", line 640, in import_ldap_users
    failed_users=failed_users)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/apps/useradmin/src/useradmin/views.py", line 776, in _import_ldap_users
    return _import_ldap_users_info(connection, user_info, sync_groups, import_by_dn, server, failed_users=failed_users)
  File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/apps/useradmin/src/useradmin/views.py", line 823, in _import_ldap_users_info
    ldap_config = desktop.conf.LDAP.LDAP_SERVERS.get()[server] if server else desktop.conf.LDAP
KeyError: u'LDAP'

[#8]

Any solution to this problem? We get the same error after an upgrade. It worked before.

[#9]

@Lanes,

Can you confirm what error you are seeing and what version of CDH you are using?

I think you are referring to this exception in the runcpserver.log:

File "/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue/apps/useradmin/src/useradmin/views.py", line 823, in _import_ldap_users_info
    ldap_config = desktop.conf.LDAP.LDAP_SERVERS.get()[server] if server else desktop.conf.LDAP
KeyError: u'LDAP'

If so, that is a known issue (Internal Cloudera Jira CDH-62230).

It is fixed in CDH 5.14.3 (not fixed in 5.15.0, though.)

There is a workaround of using Hue's ability to configure more than one LDAP server.  The code in question only fails if [[ldap_servers]] is not configured, so you can use the following type of configuration in your Hue Service safety valve.

Here is an example of what that might look like if you were using search/bind:

[desktop]
[[ldap]]
sync_groups_on_login=true
create_users_on_login=true
[[[ldap_servers]]]
[[[[LDAP]]]]
ldap_url=ldap://ldap.example.com
search_bind_authentication=true
base_dn="ou=users,dc=ad,dc=example,dc=com"
bind_dn="admin@ad.example.com"
bind_password_script={{CMF_CONF_DIR}}/altscript.sh sec-5-bind_password
[[[[[users]]]]]
user_filter="(objectClass=user)"
user_name_attr="sAMAccountName"
[[[[[groups]]]]]
group_filter="(objectClass=group)"
group_name_attr="cn"
group_member_attr="member"

Basically, you can adapt your current LDAP configuration and move it under [[[ldap_servers]]].

If you need help doing this, post your current configuration and we can lend a hand.

[#10]

We updated to CDH 5.15.0...

But the workaround works!

Thank you very much!