Hi, thanks for the detailed explanation. I truncated it intentionally to avoid some senstivie data exposure. The idp xml file was built using an online tool, I just put there the entity id, url and IDP certificate. But I did not enter any key, as Azure AD does not provide (at least) on the GUI any option to download a key. What I can try is to take the whole Federation xml file and put it into the HUE server /tmp/idp.xml ... View more

NameError: name 'appNameTEST' is not defined -> that is a syntax error, python does not know any variable with this name ... View more

It is hard to tell from this what can be the problem. Can you post the spark logs, do you have access to the Spark job UI? Do you get some error messages? ... View more

I have set to a debug level, but no more information was in the log. But then I changed the Reply URL (Assertion Consumer Service URL) to  https://<HUE_HOST>:8888/saml2/acs   and the log indicates a missing key   [20/Nov/2018 03:38:14 -0800] response DEBUG response: <?xml version='1.0' encoding='UTF-8'?> <samlp:Response xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Destination="<HUE_HOST>:8888/saml2/acs/" ..... [20/Nov/2018 03:38:14 -0800] entity DEBUG XMLSTR: <samlp:Response ID="_826d97f2-9226-4d27-b550-f4e53829ac75" Version="2.0" IssueInstant="2018-11-20T11:38:13.783Z" Destination="<HUE_HOST>:8888/saml2/acs/" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol">.... [20/Nov/2018 03:38:14 -0800] response INFO status: <?xml version='1.0' encoding='UTF-8'?> <samlp:Status xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"><samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success" /></samlp:Status> [20/Nov/2018 03:38:14 -0800] response DEBUG ***Unencrypted assertion*** [20/Nov/2018 03:38:14 -0800] response DEBUG signed [20/Nov/2018 03:38:14 -0800] sigver DEBUG ==== Certs from metadata ==== None: [] ==== [20/Nov/2018 03:38:14 -0800] response ERROR correctly_signed_response: None [20/Nov/2018 03:38:14 -0800] client_base ERROR XML parse error: None [20/Nov/2018 03:38:14 -0800] views ERROR SAML Identity Provider is not configured correctly: certificate key is missing! Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/djangosaml2-0.16.4-py2.7.egg/djangosaml2/views.py", line 254, in assertion_consumer_service response = client.parse_authn_request_response(xmlstr, BINDING_HTTP_POST, outstanding_queries) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/client_base.py", line 597, in parse_authn_request_response binding, **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/entity.py", line 1172, in _parse_response response = response.verify(keys) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/response.py", line 1017, in verify if self.parse_assertion(keys): File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/response.py", line 929, in parse_assertion if not self._assertion(assertion, False): File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/response.py", line 787, in _assertion self.xmlstr) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/sigver.py", line 1674, in check_signature id_attr=id_attr, must=must, issuer=issuer) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/sigver.py", line 1614, in _check_signature raise MissingKey("%s" % issuer) MissingKey: None [20/Nov/2018 03:38:14 -0800] middleware INFO Processing exception: : Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py", line 112, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/transaction.py", line 371, in inner return func(*args, **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/views/decorators/http.py", line 41, in inner return func(request, *args, **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/views/decorators/csrf.py", line 57, in wrapped_view return view_func(*args, **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/djangosaml2-0.16.4-py2.7.egg/djangosaml2/views.py", line 272, in assertion_consumer_service return fail_acs_response(request) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/djangosaml2-0.16.4-py2.7.egg/djangosaml2/utils.py", line 85, in fail_acs_response return failure_function(request, *args, **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/djangosaml2-0.16.4-py2.7.egg/djangosaml2/acs_failures.py", line 22, in exception_failure raise exc_class PermissionDenied     But for Azure AD there is no option to download a key.  ... View more

I dont think 5.16 is officially out (although it is available on download site) ... View more

Hi @bgooley,   I have installed the HUE service via CM, then in advance I prepared the following: - installed xmlsec1 and xmlsec1-openssl packages on Centos 7.5 - exported the certificate for SAML signing (Azure AD SSO) in base64 format - created the idp.xml file  - /tmp/idp.xml - stored the cerrtificate in base64 format as pem file /tmp/idp.pem   Set on the IDP the URL for login (https://<hue-fqdn>:8888)   Set on advanced properties the following:   [[auth]] backend=libsaml.backend.SAML2Backend [libsaml] xmlsec_binary=/usr/bin/xmlsec1 metadata_file=/tmp/idp.xml cert_file=/tmp/idp.pem   Restarted the HUE service. Then I try to initiate the login process and get this:   [20/Nov/2018 02:36:31 -0800] middleware INFO Redirecting to login page: / [20/Nov/2018 02:36:31 -0800] access INFO 10.85.239.134 -anon- - "POST / HTTP/1.1" -- login redirection [20/Nov/2018 02:36:31 -0800] access INFO 10.85.239.134 -anon- - "POST / HTTP/1.1" returned in 0ms [20/Nov/2018 02:36:32 -0800] client INFO destination to provider: https://login.microsoftonline.com/<AZURE_CLIENT_ID>/saml2 [20/Nov/2018 02:36:32 -0800] entity INFO REQUEST: <?xml version='1.0' encoding='UTF-8'?> <samlp:AuthnRequest xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" AssertionConsumerServiceURL="https://<HUE_HOST>:8888/saml2/acs/" Destination="https://login.microsoftonline.com/<AZURE_CLIENT_ID>/saml2" ID="xxxxxxxxxxxxxxxx" IssueInstant="2018-11-20T10:36:32Z" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Version="2.0"><saml:Issuer Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">https://<HUE_HOST>:8888/saml2/metadata/</saml:Issuer><samlp:NameIDPolicy AllowCreate="false" Format="urn:oasis:names:tc:SAML:2.0:nameid-format:persistent" /></samlp:AuthnRequest> [20/Nov/2018 02:36:32 -0800] client INFO AuthNReq: <?xml version='1.0' encoding='UTF-8'?> <samlp:AuthnRequest xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" AssertionConsumerServiceURL="https://<HUE_HOST>:8888/saml2/acs/" Destination="https://login.microsoftonline.com/<AZURE_CLIENT_ID>/saml2" ID="xxxxxxxxxxxxxxxx" IssueInstant="2018-11-20T10:36:32Z" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Version="2.0"><saml:Issuer Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">https://<HUE_HOST>:8888/saml2/metadata/</saml:Issuer><samlp:NameIDPolicy AllowCreate="false" Format="urn:oasis:names:tc:SAML:2.0:nameid-format:persistent" /></samlp:AuthnRequest> [20/Nov/2018 02:36:32 -0800] entity INFO HTTP REDIRECT [20/Nov/2018 02:36:32 -0800] middleware INFO Processing exception: 'NoneType' object has no attribute 'get_signer': Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py", line 112, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/transaction.py", line 371, in inner return func(*args, **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/djangosaml2-0.16.4-py2.7.egg/djangosaml2/views.py", line 175, in login binding=binding, sign=False, sigalg=sigalg) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/client.py", line 76, in prepare_for_authenticate **kwargs) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/client.py", line 129, in prepare_for_negotiated_authenticate relay_state, **args) File "/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/pysaml2-4.4.0-py2.7.egg/saml2/entity.py", line 231, in apply_binding signer = self.sec.sec_backend.get_signer(kwargs['sigalg']) AttributeError: 'NoneType' object has no attribute 'get_signer' Let me please now how to set HUE logs into DEBUG level, I have not found any settings in CM for this, nor any advanced snippet. Thanks ... View more

Maybe off topic: but even with LDAP how do you want to implement security? (PErmissions on tables, databases). I suspect your EMR is not using Kerberos right? ... View more