Member since
09-19-2016
11
Posts
1
Kudos Received
0
Solutions
03-23-2017
04:01 PM
After investigations, we see that the problem is not related to Hue. When using Tez from command line, it's working fine. When using beeline from command line, we connect using JDBC url, and then we get the same error as from Hue: org.apache.tez.dag.api.TezUncheckedException:Invalid configuration of tez jars, tez.lib.uris isnotdefinedin the configuration
... View more
03-21-2017
03:32 PM
We also run an original version of Hue 2.6 that has exactly the same problem.
... View more
03-21-2017
03:28 PM
Hello community, We are running HDP 2.5 with a kerberos setup. We have installed and configured Hue 3.12 and all editors are working fine. From Hue Hive editor, Map Reduce is working fine: SET hive.execution.engine=mr;
SELECT count(*) FROM <my_table>; But if I execute the same query using Tez: SET hive.execution.engine=tez;
SELECT count(*) FROM <my_table>; I am getting the following error: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration The property tez.lib.uris is correctly set in Ambari to tez.lib.uris = /hdp/apps/${hdp.version}/tez/tez.tar.gz Tez is indeed correctly configured since from the command line, both queries are returning successfully. So it's a Hue + Hive + Tez configuration issue. Has anyone a hint on how to investigate this further? Thanks a lot, Tristan
... View more
- Tags:
- Cloud & Operations
- Hive
- hue
- tez
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
12-02-2016
02:11 PM
Unfortunately, we never managed to find a work around, and Hortonworks is not supporting Hue. Our plan is to use a much recent version of Hue.
... View more
09-20-2016
12:57 PM
Here is attached the hue.ini (I have blanked server names) hueini.txt
... View more
09-20-2016
09:47 AM
Thanks @hkropp. I'm looking at the suggested options. What about plain LDAP login configuration on the nodes, could it be a simple solution too?
... View more
09-20-2016
06:58 AM
Hello, We have a working HDP 2.4 Kerberos enabled working, with Hue and from command line too. Following to the documentation, we have created Unix user mappings between the nodes and Active Directory. Users can login to a utility machine and kinit to login. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_Security_Guide/content/create_mappings_betw_principals_and_unix_usernames.html It's working just fine. My question is about the management of those users: everytime we want to grand access to a new user, we need to go ahead and create those users on all nodes, which is not very practical. What are the best practises for that? script user accounts creation? Enrol all node machines as domain machines so that AD users can login? Configure the unix machine to accept login using LDAP against AD? Others? Thanks a lot in advance for ideals and experiences. Tristan
... View more
Labels:
09-20-2016
06:48 AM
Thanks for the suggestion. I am confirming that history_server_api_url is correct. Looking at the logs you are referring reveals the following when looking at a Map Reduce job: [20/Sep/2016 08:45:15 +0000] access INFO 10.50.0.59 trifil - "GET /jobbrowser/jobs/application_1474295224072_0013 HTTP/1.1"
[20/Sep/2016 08:45:15 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
[20/Sep/2016 08:45:15 +0000] middleware INFO Processing exception: Could not find job application_1474295224072_0013.: Traceback (most recent call last):
File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py", line 100, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/hue/apps/jobbrowser/src/jobbrowser/views.py", line 62, in decorate
raise PopupException(_('Could not find job %s.') % jobid, detail=e)
PopupException: Could not find job application_1474295224072_0013. When looking at a TEZ job (which is working), I am getting the following logs: [20/Sep/2016 08:47:21 +0000] access INFO 10.50.0.59 trifil - "GET /jobbrowser/jobs_ats/application_1474295224072_0005 HTTP/1.1"
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 404 response
[20/Sep/2016 08:47:21 +0000] access INFO 10.50.0.59 trifil - "GET /jobbrowser/jobs_ats/application_1474295224072_0005 HTTP/1.1"
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
[20/Sep/2016 08:47:21 +0000] kerberos_ ERROR handle_other(): Mutual authentication unavailable on 200 response
... View more
09-20-2016
06:41 AM
Hello, From https://mycluster_hue:8000/jobbrowser/, the duration column is showing 1things like 7064d:6h:32m:31s (which is probably the elapsed time from 01/01/1970) When I go to the job detail screen the start date, end date and duration are correct, which confirms the time server are OK and the job meta data are correct too. Has anyone the same problem, and a tip to fix this? Thanks a lot for the community help, Tristan Job browser: Job detail:
... View more
09-19-2016
03:28 PM
1 Kudo
On HDP 2.4, we're using the standard Hue with security on (Hue 2.6.1-258) We have map reduce, pig, hive and spark running successfully from the command line and Kerberos. In Hue, we have also everything working fine, expect when we submit a Map Reduce job: the job is submitted and working, but when looking at it from the job history, we are getting the following error: Job job_1474295224072_0004 could not be found: <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 401 Authentication required</title> </head> <body><h2>HTTP ERROR 401</h2> <p>Problem accessing /ws/v1/history/mapreduce/jobs/job_1474295224072_0004. Reason: <pre> Authentication required</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body> </html> (error 401) When submitting TEZ or Spark jobs, there is no issue, we can see the job details ( it's actually going to another REST url to query for information). The job browser problem is only limited to Map Reduce jobs. When tracking down the issue, we see the following URL in the Hue logs for the job history: http_client DEBUG REST invocation: curl -X GET -H Accept: application/json 'http://my_cluster:19888/ws/v1/history/mapreduce/jobs/job_1474295224072_0004' Which is wrong. It should contain a --negotiate such as: http_client DEBUG REST invocation: curl -X GET --negotiate -H Accept: application/json 'http://my_cluster:19888/ws/v1/history/mapreduce/jobs/job_1474295224072_0004' Looking at the source code shows no indication of any kind of support for that. So my question is: is Hue shipped in HDP 2.4 capable to use Kerberos for this job browser? If not, any hints on what is wrong in our setup?
... View more
09-19-2016
03:20 PM
Hello, we are having exactly the same problem. We have map reduce, pig, hive and spark running successfully from the command line. Our setup works.
In Hue, we have also everything working fine, expect when we submit a Map Reduce job: the job is working, but when looking at it from the job history, we are getting the same error. When submitting TEZ or Spark jobs, we can see the job details. When tracking down the issue, we see that in the Hue logs that for the job history:
http_client DEBUG REST invocation: curl -X GET -H Accept: application/json 'http://my_cluster:19888/ws/v1/history/mapreduce/jobs/job_1474295224072_0004' Which is wrong. It should contain a --negotiate such as: http_client DEBUG REST invocation: curl -X GET --negotiate -H Accept: application/json 'http://my_cluster:19888/ws/v1/history/mapreduce/jobs/job_1474295224072_0004' Looking at the source code shows no indication of any kind of support for that. So my question is: is Hue shipped in HDP 2.4 capable to use Kerberos for this job browser?
... View more