Member since
05-27-2016
22
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2032 | 12-23-2020 12:05 AM | |
2212 | 12-18-2020 05:57 AM |
12-23-2020
12:05 AM
LLAP internally uses MAP-REDUCE shuffle handler because inside TEZ code we have a hard-coded constant TEZ_SHUFFLE_HANDLER_SERVICE_ID = "mapreduce_shuffle". https://github.com/apache/tez/blob/master/tez-api/src/main/java/org/apache/tez/dag/api/TezConstants.java So, when we are using tez_shuffle with LLAP its giving ‘NullpointerException’.
... View more
12-18-2020
05:57 AM
why don't you write 20 insert into query in append mode in the same sql file ';' separated, In this case you don't have to use union. And probably won't encounter heap space issue.
... View more
12-18-2020
05:49 AM
After Setup for the Tez Shuffle Handler following instructions on : https://tez.apache.org/shuffle-handler.html I'm getting below error in query execution: Vertex failed, vertexName=Map 2, vertexId=vertex_1608273679503_0002_2_01, diagnostics=[Task failed, taskId=task_1608273679503_0002_2_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1608273679503_0002_2_01_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:363) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:266) ... 15 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async Initialization failed. abortRequested=false at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:461) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:395) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:568) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:520) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:381) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:335) ... 16 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:118) at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache$1.call(LlapObjectCache.java:143) ... 4 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:113) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:331) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:400) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:207) at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:116) ... 5 more Caused by: java.lang.NullPointerException at org.apache.tez.runtime.api.impl.TezTaskContextImpl.getServiceConsumerMetaData(TezTaskContextImpl.java:190) at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleManager.(ShuffleManager.java:264) at org.apache.tez.runtime.library.input.UnorderedKVInput.start(UnorderedKVInput.java:146) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:109) ... 9 more Can anyone help here please?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Tez
10-30-2020
12:09 AM
Thank you! This was helpful.
... View more
10-28-2020
12:00 PM
I want to set logging level of hiveserver2Interactive.log to DEBUG in HDP-3.1. could anyone help here?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive
04-20-2017
10:39 AM
what is the solution?
... View more
04-20-2017
10:27 AM
how was the issue resolved? could you please provide some information.
... View more
04-20-2017
08:45 AM
command "hdfs fsck / -delete" worked for me.
... View more
01-24-2017
11:35 AM
@Kuldeep Kulkarni, Yes it was a problem with my rpm command, that's why yum hung. I rebuild rpmdb and everything working fine now. thanks a ton. 🙂
... View more
01-24-2017
06:00 AM
@Kuldeep Kulkarni
I encountered similar issue with HDFS_CLIENT, I've successfully deleted the client using curl call as mentioned in your post.
But, Installation of HDFS_CLIENT stuck at:
2017-01-24 08:44:43,873 - Package['unzip'] {}
And installation failed with following error message:
stderr: /var/lib/ambari-agent/data/errors-2631.txt
Python script has been killed due to timeout after waiting 1800 secs Could you please suggest me some pointers?
... View more