Reply
Contributor
Posts: 25
Registered: ‎09-04-2018

Error 500 in Hue while adding new groups, users from Open LDAP

Hello!

 

When I am trying to add users or groups from LDAP, or sync groups from LDAP in Hue web UI, I get error 500  ({"auth": false}).

 

27/Dec/2018 11:03:05 +0300] access       WARNING  10.160.0.3 admin - "POST /useradmin/users/sync_ldap_users_groups HTTP/1.1" -- Failed login for user: None

 

It worked perfectly earlier.

Posts: 1,047
Topics: 1
Kudos: 263
Solutions: 131
Registered: ‎04-22-2014

Re: Error 500 in Hue while adding new groups, users from Open LDAP

@Paulina,

 

Can you supply more logging around the WARNING line?  The POST itself indicates there could be some sort of issue, but without context, it is tough to understand what might be happening.

Perhaps include lines at least 1 minute before and after the WARNING just to make sure.

Highlighted
Contributor
Posts: 25
Registered: ‎09-04-2018

Re: Error 500 in Hue while adding new groups, users from Open LDAP

hello @bgooley

 

the log before

 

[09/Jan/2019 14:34:39 +0300] access INFO 10.160.0.2 maslova - "GET /filebrowser/ HTTP/1.1"

[09/Jan/2019 14:34:39 +0300] connectionpool INFO Resetting dropped connection: sspesrch02v.sec.oteco

[09/Jan/2019 14:34:39 +0300] connection WARNING /usr/lib/hue/build/env/lib/python2.7/site-packages/requests-2.10.0-py2.7.egg/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for sspesrch02v.sec.oteco has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)

SubjectAltNameWarning

 

[09/Jan/2019 14:34:39 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:34:39 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:34:41 +0300] access INFO 10.160.0.2 maslova - "GET /filebrowser/view=/data HTTP/1.1"

[09/Jan/2019 14:34:41 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:34:41 +0300] connectionpool INFO Resetting dropped connection: sspesrch02v.sec.oteco

[09/Jan/2019 14:34:41 +0300] connection WARNING /usr/lib/hue/build/env/lib/python2.7/site-packages/requests-2.10.0-py2.7.egg/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for sspesrch02v.sec.oteco has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)

SubjectAltNameWarning

 

[09/Jan/2019 14:34:46 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users HTTP/1.1"

[09/Jan/2019 14:34:46 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:34:46 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:34:47 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:34:54 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:34:54 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:34:54 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:34:54 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:35:02 +0300] access INFO 10.160.0.2 maslova - "POST /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:35:03 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users HTTP/1.1"

[09/Jan/2019 14:35:03 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:35:03 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:35:03 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:35:18 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:35:18 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:35:18 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:35:19 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:35:28 +0300] access INFO 10.160.0.2 maslova - "POST /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:35:29 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users HTTP/1.1"

[09/Jan/2019 14:35:29 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:35:29 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:35:29 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:35:41 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:35:41 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:35:41 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:35:41 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:35:49 +0300] access INFO 10.160.0.2 maslova - "POST /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:35:49 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users HTTP/1.1"

[09/Jan/2019 14:35:49 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:35:49 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:35:50 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:36:02 +0300] access INFO 10.160.0.2 maslova - "GET /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:36:02 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:36:02 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:36:02 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:36:09 +0300] access INFO 10.160.0.2 maslova - "POST /useradmin/users/add_ldap_users HTTP/1.1"

[09/Jan/2019 14:36:09 +0300] ldap_access WARNING Last name is truncated to 30 characters for [<User: bigdata0>].

[09/Jan/2019 14:36:09 +0300] middleware INFO Processing exception: invalid byte sequence for encoding "UTF8": 0xd0 0x20

: Traceback (most recent call last):

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py", line 112, in get_response

response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/transaction.py", line 371, in inner

return func(*args, **kwargs)

File "/usr/lib/hue/apps/useradmin/src/useradmin/views.py", line 476, in add_ldap_users

users = import_ldap_users(connection, username_pattern, False, import_by_dn, failed_users=failed_ldap_users)

File "/usr/lib/hue/apps/useradmin/src/useradmin/views.py", line 645, in import_ldap_users

failed_users=failed_users)

File "/usr/lib/hue/apps/useradmin/src/useradmin/views.py", line 781, in _import_ldap_users

return _import_ldap_users_info(connection, user_info, sync_groups, import_by_dn, server, failed_users=failed_users)

File "/usr/lib/hue/apps/useradmin/src/useradmin/views.py", line 819, in _import_ldap_users_info

user.save()

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py", line 545, in save

force_update=force_update, update_fields=update_fields)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py", line 573, in save_base

updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py", line 635, in _save_table

forced_update)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py", line 679, in _do_update

return filtered._update(values) > 0

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/query.py", line 510, in _update

return query.get_compiler(self.db).execute_sql(None)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/sql/compiler.py", line 980, in execute_sql

cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/sql/compiler.py", line 786, in execute_sql

cursor.execute(sql, params)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/backends/util.py", line 53, in execute

return self.cursor.execute(sql, params)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/utils.py", line 99, in __exit__

six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/backends/util.py", line 53, in execute

return self.cursor.execute(sql, params)

DataError: invalid byte sequence for encoding "UTF8": 0xd0 0x20

 

 

[09/Jan/2019 14:36:09 +0300] decorators INFO AXES: Calling decorated function: dt_login

[09/Jan/2019 14:36:09 +0300] decorators INFO args: (True,)

[09/Jan/2019 14:36:09 +0300] access WARNING 10.160.0.2 maslova - "POST /useradmin/users/add_ldap_users HTTP/1.1" -- Failed login for user: None

[09/Jan/2019 14:36:09 +0300] decorators INFO AXES: Repeated login failure by 10.160.0.2. Updating access record. Count = 18

[09/Jan/2019 14:36:13 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:36:44 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:37:14 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:37:44 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

[09/Jan/2019 14:38:14 +0300] access INFO 10.160.0.2 maslova - "POST /jobbrowser/jobs/ HTTP/1.1"

Posts: 954
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Error 500 in Hue while adding new groups, users from Open LDAP

Hi @Paulina,

 

Thanks for posting more logs requested. That is very helpful.

 

The log shows the error message below:

File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/backends/util.py", line 53, in execute

return self.cursor.execute(sql, params)

DataError: invalid byte sequence for encoding "UTF8": 0xd0 0x20

 

Usually above error indicates the encoding of the data you are trying to insert into the backend database is not matching what database is looking for.

 

Quick questions:

1) What is your backend database for Hue? Is it postgres?

2) Does the particular user name contain any special characters?

3) Does the issue happen to other users?

 

Thanks,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Contributor
Posts: 25
Registered: ‎09-04-2018

Re: Error 500 in Hue while adding new groups, users from Open LDAP

[ Edited ]

Thank you for your answer, @lwang!

 

1) Yes, postgres, UTF8.

2) No. We are trying to sync users like bigdata0, bigdata34 etc.

 

Here is user in ldif:

 

dn: uid=bigdata34,ou=users,dc=sec,dc=oteco

objectClass: extensibleObject

objectClass: posixAccount

objectClass: top

objectClass: shadowAccount

objectClass: account

cn: bigdata34

gidNumber: 1011

homeDirectory: /data/users/bigdata34

uid: bigdata34

uidNumber: 1011

loginShell: /bin/bash

mail: somebody@gmail.com

sn:: 0JzQuNGF0LDQu9C10L3QutC+INCQ0LvQtdC60YHQtdC5

userPassword:: e1NTSEF9dzhJYjlqZE1RZHZPVURETHNMUzYvbnl3WnNhb3laZnA=

 

sn is in cyrillic characters

 

3) This happens on some users. For example, bigdata34 goes well even with cyrillic characters, bigdata70 shows error.

 

dn: uid=bigdata70,ou=users,dc=sec,dc=oteco

objectClass: extensibleObject

objectClass: posixAccount

objectClass: top

objectClass: shadowAccount

objectClass: account

cn: bigdata70

gidNumber: 1016

homeDirectory: /data/users/bigdata70

uid: bigdata70

uidNumber: 1016

loginShell: /bin/bash

mail: ххххххх@gmail.com

sn:: 0KHQsNGA0LLQsNGC0LTQuNC90L7QsiDQkNC70LXQutGB0LDQvdC00YA=

userPassword:: e01ENX1xeU03YUM3RFZXU09lSkhtYkZRWkd3PT0=

 

 

Posts: 954
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Error 500 in Hue while adding new groups, users from Open LDAP

Hi @Paulina,

 

Based on the log file which complains the user " [<User: bigdata0>]", can you please send us the ldapsearch output for the user bigdata0?

 

Thanks,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Contributor
Posts: 25
Registered: ‎09-04-2018

Re: Error 500 in Hue while adding new groups, users from Open LDAP

[ Edited ]

Dear @lwang!

 

ldapsearch results below

 

# extended LDIF
#
# LDAPv3
# base <dc=sec,dc=oteco> with scope subtree
# filter: (uid=bigdata0)
# requesting: ALL
#

# bigdata0, users, sec.oteco
dn: uid=bigdata0,ou=users,dc=sec,dc=oteco
uid: bigdata0
objectClass: account
objectClass: shadowAccount
objectClass: top
objectClass: posixAccount
objectClass: extensibleObject
loginShell: /bin/bash
userPassword:: e01ENX1JQ3k1WXF4WkIxdVdTd2NWTFNOTGNBPT0=
sn:: 0JrQuNGB0LvQvtCy0LAg0JLQsNC70LjQvdGC0LjQvdCw
gidNumber: 1017
uidNumber: 1017
cn: bigdata0
homeDirectory: /data/users/bigdata0
mail: ххххххх@gmail.com

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Posts: 954
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Error 500 in Hue while adding new groups, users from Open LDAP

Hi @Paulina,

 

What is your CDH version? Would like to try this out in house to see whether we can replicate the issue.

 

Thanks!

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Contributor
Posts: 25
Registered: ‎09-04-2018

Re: Error 500 in Hue while adding new groups, users from Open LDAP

Dear @lwang!

 

We use

Version: Cloudera Express 5.9.3 (#6 built by jenkins on 20170627-1506 git: 23506bb4e114dd460bdac64c57a672e6be631907)

Java VM Name: Java HotSpot(TM) 64-Bit Server VM

Java VM Vendor: Oracle Corporation

Java Version: 1.7.0_67

OpenLDAP 2.4.44

Thank you for your help!

 

Posts: 954
Kudos: 30
Solutions: 18
Registered: ‎05-27-2014

Re: Error 500 in Hue while adding new groups, users from Open LDAP

Hi @Paulina,

 

I would like to let you know I am still in the middle of setting up a test environment. Will get back to you when I have some results.

 

Thanks,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum