Member since
09-21-2018
6
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3772 | 10-12-2018 08:41 PM |
10-12-2018
08:41 PM
Below Statement worked for me finally .
hive> desc formatted testdb.table col_name partition (data_dt='20180715');
... View more
10-05-2018
03:58 PM
@Sindhu, Thanks for the reply. I tried the same and it wont return any values. But the compute statistics command worked fine though hive> analyze table testdb.table partition(data_dt) compute statistics for columns order_dispatch_diff;
Query ID = anil_20181004181226_ed6afd54-fed8-4601-8b54-29214a8dbfc3
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1537195062060_1102639)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 62 62 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 101.08 s
--------------------------------------------------------------------------------
OK
Time taken: 108.142 seconds
hive> desc extended testdb.table order_dispatch_diff;
OK
order_dispatch_diff int from deserializer
Time taken: 0.041 seconds, Fetched: 1 row(s)
... View more
10-04-2018
05:44 PM
Trying to see statistics on a particular column. I executed the ANALYZE command first and then tried to see the stats by DESCRIBE FORMATTED <table_name> <col_name>. I cant see any values in this. Any idea why its not showing any values? I tried MSCK and analyzed the table again and checked for stats. No luck so far. Thanks Anil
... View more
Labels:
- Labels:
-
Apache Hive
10-03-2018
04:38 PM
Worked like a charm!. Thank you.
... View more
10-02-2018
07:08 PM
Hive Query fails with the below error when running in Tez Mode. My application dont have option to alter the execution mode. So tez is the only option. Is this a bug from Tez ? Query run fine in MR mode when tried from shell.. Error: --------------------------------------------------------------------------------
Map 1 ..... RUNNING 494 278 0 216 343 0
Reducer 2 INITED 62 0 0 62 0 0
--------------------------------------------------------------------------------
VERTICES: 00/02 [=============>>-------------] 50% ELAPSED TIME: 41.84 s
--------------------------------------------------------------------------------
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1537195062060_938252_1_00, diagnostics=[Task failed, taskId=task_1537195062060_938252_1_00_000367, diagn=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOEon: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritable is not class org.apache.hadoop.io.BytesWritable
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: java.io.IOException: wrong key class: org.apache.hadoop.io.NullWritablet class org.apache.hadoop.io.BytesWritable
at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:71)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
... 14 more
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Tez
09-21-2018
08:13 PM
As part of the upgrade process , we are taking backup using "fsck / -files -blocks -locations > fsck.log" Size of fsimage was 12GB .. but the size of this log is going beyond 50 GB.. How can we recover fsimage from this fsck report? Is it worth to spend time to take this backup? Thanks Anil
... View more
Labels: