Member since
06-06-2016
185
Posts
12
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2691 | 07-20-2016 07:47 AM | |
2256 | 07-12-2016 12:59 PM |
08-04-2016
02:06 PM
@Michael Young Thanks you so much..now only i got exact answer for my question since long i am waiting for this confirmation .But i have small concern ,according my understanding when jobs are completed the temp files are automatically deleted ,why this files there in tmp files? can please explain in details.
... View more
08-04-2016
01:25 PM
hdfs@ABCHADOOP1-15-2:/root> hadoop fs -du -h /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086 7.9 T /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001
hdfs@ABCHADOOP1-15-2:/root> hadoop fs -du -h /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001
687.8 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000000_0
687.4 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000001_0
686.9 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000002_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000003_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000004_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000005_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000006_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000007_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000008_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000009_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000010_0
653.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000011_0
73.0 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000012_0
34.2 G /tmp/hive-beeswax-abic720prod/hive_2016-04-12_10-09-43_383_5647515039912810955-1086/-ext-10001/000013_0
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
-
Cloudera Hue
08-02-2016
12:58 PM
@mqureshi Thank you so much i really appreciate your response.. .user access in DEV is cleared but only still issue there PROD... Its has below error log 16/08/02 01:09:12 INFO tez.TezSessionState: User of session id baacb86a-e54d-4fae-8c16-fff9834d4d8a is y919122
16/08/02 01:11:03 INFO tez.DagUtils: Jar dir is null/directory doesn't exist. Choosing HIVE_INSTALL_DIR - hdfs:/user/y919122/.hiveJars
16/08/02 01:12:52 ERROR thrift.ProcessFunction: Internal error processing query
java.lang.OutOfMemoryError: Java heap space its is says that java heap space ..but i could run same query in CLI so what should i do in HUE(hive beewax)?
... View more
08-01-2016
02:39 PM
Ok ..but i could able to run same query from different users(all are new users) but error comes with one single user(old user) ..all are in same group as well
... View more
08-01-2016
12:40 PM
@ mqureshi Previously i had error: Internal error processing query but now i reciving time out error i post the error.log & beeswax_server.out TApplicationException: Internal error processing query [30/Jul/2016 22:46:05 +0000] thrift_util ERROR Thrift saw exception (this may be expected). Traceback (most recent call last): File "/usr/lib/hue/desktop/core/src/desktop/lib/thrift_util.py", line 348, in wrapper ret = res(*args, **kwargs) File "/usr/lib/hue/apps/beeswax/gen-py/beeswaxd/BeeswaxService.py", line 142, in query return self.recv_query() File "/usr/lib/hue/apps/beeswax/gen-py/beeswaxd/BeeswaxService.py", line 158, in recv_query raise x TApplicationException: Internal error processing query [01/Aug/2016 00:16:49 +0000] views ERROR Failed to send query completion notification via e-mail: QueryHistory matching query does not exist. [01/Aug/2016 01:09:25 +0000] views ERROR Failed to send query completion notification via e-mail: QueryHistory matching query does not exist.
beeswax_server.out 16/08/01 00:48:15 INFO tez.TezSessionState: User of session id 5cb11a09-71fb-4970-95d6-b1ac73a0c804 is y919122 16/08/01 00:53:25 INFO tez.DagUtils: Jar dir is null/directory doesn't exist. Choosing HIVE_INSTALL_DIR - hdfs:/user/y919122/.hiveJars 16/08/01 00:56:33 ERROR thrift.ProcessFunction: Internal error processing query java.lang.OutOfMemoryError: Java heap space 16/08/01 00:56:33 INFO log.PerfLogger: </PERFLOG method=Driver.execute start=1470036998452 end=1470038193846 duration=1195394 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 00:56:34 INFO tez.DagUtils: Localizing resource because it does not exist: file:/usr/lib/hive/lib/hive-exec-0.13.0.2.1.2.2-516.jar to dest: hdfs://ABIHADOOP1/user/y919122/.hiveJars/hive-exec-0.13.0.2.1.2.2-516-82bbaf519912703e421e55e9826552910417b9fcfae3f865e2e07f826a9f260a.jar 16/08/01 00:56:50 INFO tez.DagUtils: Looks like another thread is writing the same file will wait. 16/08/01 00:56:50 INFO tez.DagUtils: Number of wait attempts: 5. Wait interval: 5000 16/08/01 00:57:06 INFO tez.DagUtils: Resource modification time: 1469367036707 16/08/01 00:57:08 INFO tez.TezSessionState: Opening new Tez Session (id: 5cb11a09-71fb-4970-95d6-b1ac73a0c804, scratch dir: hdfs://ABIHADOOP1/tmp/hive-beeswax-y919122/_tez_session_dir/5cb11a09-71fb-4970-95d6-b1ac73a0c804) 16/08/01 00:57:49 INFO client.RMProxy: Connecting to ResourceManager at stlpr8712/39.6.64.3:8050 16/08/01 00:59:25 INFO impl.YarnClientImpl: Submitted application application_1443521267046_220241 16/08/01 00:59:25 INFO log.PerfLogger: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver> 16/08/01 00:59:25 INFO ql.Driver: Starting command: show databases 16/08/01 01:04:22 INFO impl.TimelineClientImpl: Timeline service address: http://s11:8188/ws/v1/timeline/ 16/08/01 01:04:22 INFO hooks.ATSHook: Created ATS Hook 16/08/01 01:04:22 INFO log.PerfLogger: <PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:04:22 INFO log.PerfLogger: </PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1470038662694 end=1470038662694 duration=0 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:04:22 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit start=1470035682189 end=1470038662695 duration=2980506 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:04:22 INFO log.PerfLogger: <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:04:22 INFO log.PerfLogger: <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:04:22 INFO exec.DDLTask: results : 48 16/08/01 01:04:24 INFO log.PerfLogger: </PERFLOG method=runTasks start=1470038662695 end=1470038664516 duration=1821 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:09:25 INFO impl.TimelineClientImpl: Timeline service address: http://s1:8188/ws/v1/timeline/ 16/08/01 01:09:25 INFO hooks.ATSHook: Created ATS Hook 16/08/01 01:09:25 INFO log.PerfLogger: <PERFLOG method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:09:25 INFO log.PerfLogger: </PERFLOG method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1470038965108 end=1470038965108 duration=0 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:09:25 INFO log.PerfLogger: </PERFLOG method=Driver.execute start=1470038365875 end=1470038965108 duration=599233 from=org.apache.hadoop.hive.ql.Driver> 16/08/01 01:09:25 INFO ql.Driver: OK 16/08/01 04:22:15 ERROR thrift.ProcessFunction: Internal error processing query java.lang.OutOfMemoryError: Java heap space
... View more
07-30-2016
05:29 AM
@mqureshi Hue: 2.3.1-402 Hive: 0.13.0.2.1 I think so its support because previously until one day back i have successfully run hive queries in hue, today only i have this issue...I have same conf and versions in TEST here can run hive queries in HUE..but problem in PROD only. comes logs i just wondering which logs should check i have below logs -rw-r--r-- 1 hue hue 989989 Jul 30 00:04 access.log -rw-r--r-- 1 hue hue 999960 Jul 27 08:18 access.log.1
-rw-r--r-- 1 hue hue 999961 Jul 21 08:38 access.log.2
-rw-r--r-- 1 hue hue 999978 Jul 20 05:35 access.log.3
-rw-r--r-- 1 hue hue 27700 Sep 23 2015 beeswax_server.log
-rw-r--r-- 1 hue hue 628497762 Jul 29 22:36 beeswax_server.out
-rw-r--r-- 1 hue hue 986613 Jul 29 22:36 error.log -rw-r--r-- 1 hue hue 999246 Jun 1 11:53 error.log.1
-rw-r--r-- 1 hue hue 991759 Feb 15 09:54 error.log.2 -rw-r--r-- 1 hue hue 999264 Dec 8 2015 error.log.3
-rw-r--r-- 1 hue hue 12691 Sep 23 2015 kt_renewer.log -rw-r--r-- 1 hue hue 0 Aug 26 2014 kt_renewer.out
-rw-r--r-- 1 hue hue 65644 Jul 30 00:04 runcpserver.log
-rw-r--r-- 1 hue hue 999967 Jul 29 22:34 runcpserver.log.1
-rw-r--r-- 1 hue hue 999942 Jul 28 12:36 runcpserver.log.2
-rw-r--r-- 1 hue hue 999160 Jul 28 08:30 runcpserver.log.3
-rw-r--r-- 1 hue hue 114927 Jul 19 10:33 runcpserver.out -rw-r--r-- 1 hue hue 6603 Jun 27 05:37 shell_input.log -rw-r--r-- 1 hue hue 36501 Jun 27 05:37 shell_output.log -rw-r--r-- 1 hue hue 11758 Sep 23 2015 supervisor.log -rw-r--r-- 1 hue hue 0 Aug 26 2014 supervisor.out
... View more
07-30-2016
03:44 AM
Hi..i am using HDP 2.1.2 and Hue(beewax hive UI) ..while i run simple hive query(show databases) its show internal error processing query error But i could see all data bases and tables in hue and also i could run hive queries in command line..please help me get out of this.. Hive: 0.13.0.2.1 Tez:0.4.0.2.1
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
-
Cloudera Hue
07-25-2016
11:29 AM
Thank you rahul..how can i disable this from ranger? can you explain in wide?
... View more
07-25-2016
09:48 AM
thanks you guys ..i will check it let you know..
... View more
07-24-2016
08:19 AM
i have hdp sandbox 2.3..i login with user "sankar" and my hive warehouse directory having below file permission hadoop fs -ls /apps/hive drwxr-xr-x hive hdfs /apps/hive/warehouse Here i see that owner having: rwx; group:r-x other: r-x permission But while i login to sankar user i can easily put some data into that directory i have attached below screen short filesystem.png Can any one explain this in my understanding way...please
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive