Member since
05-27-2016
22
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
944 | 12-23-2020 12:05 AM | |
1010 | 12-18-2020 05:57 AM |
12-23-2020
12:05 AM
LLAP internally uses MAP-REDUCE shuffle handler because inside TEZ code we have a hard-coded constant TEZ_SHUFFLE_HANDLER_SERVICE_ID = "mapreduce_shuffle". https://github.com/apache/tez/blob/master/tez-api/src/main/java/org/apache/tez/dag/api/TezConstants.java So, when we are using tez_shuffle with LLAP its giving ‘NullpointerException’.
... View more
12-18-2020
05:57 AM
why don't you write 20 insert into query in append mode in the same sql file ';' separated, In this case you don't have to use union. And probably won't encounter heap space issue.
... View more
12-18-2020
05:49 AM
After Setup for the Tez Shuffle Handler following instructions on : https://tez.apache.org/shuffle-handler.html I'm getting below error in query execution: Vertex failed, vertexName=Map 2, vertexId=vertex_1608273679503_0002_2_01, diagnostics=[Task failed, taskId=task_1608273679503_0002_2_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1608273679503_0002_2_01_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:363) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:266) ... 15 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async Initialization failed. abortRequested=false at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:461) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:395) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:568) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:520) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:381) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:335) ... 16 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:118) at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache$1.call(LlapObjectCache.java:143) ... 4 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:113) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:331) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:400) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:207) at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:116) ... 5 more Caused by: java.lang.NullPointerException at org.apache.tez.runtime.api.impl.TezTaskContextImpl.getServiceConsumerMetaData(TezTaskContextImpl.java:190) at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleManager.(ShuffleManager.java:264) at org.apache.tez.runtime.library.input.UnorderedKVInput.start(UnorderedKVInput.java:146) at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:109) ... 9 more Can anyone help here please?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Tez
10-30-2020
12:09 AM
Thank you! This was helpful.
... View more
10-28-2020
12:00 PM
I want to set logging level of hiveserver2Interactive.log to DEBUG in HDP-3.1. could anyone help here?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive
04-20-2017
10:39 AM
what is the solution?
... View more
04-20-2017
10:27 AM
how was the issue resolved? could you please provide some information.
... View more
04-20-2017
08:45 AM
command "hdfs fsck / -delete" worked for me.
... View more
04-18-2017
05:24 AM
Why solr service not present in HDP2.4 stack? Although it is shown(in the above image) that solr is available since HDP2.1. Can anyone please help why I'am not able to find solr service in HDP stack.
... View more
Labels:
03-10-2017
04:36 PM
Just to add, if any of the component is installed then package of that component will not reinstall, so in case any file of that component is removed from /usr/hdp directory, then start of the service for that particular component will fail. Correct me if I'm wrong.
... View more
03-09-2017
05:23 PM
@Yu Song: you can check which service is giving this error. after that you need to remove the sym link of that service from /usr/hdp/current/ directory and restart the installation of that service.
... View more
01-24-2017
11:35 AM
@Kuldeep Kulkarni, Yes it was a problem with my rpm command, that's why yum hung. I rebuild rpmdb and everything working fine now. thanks a ton. 🙂
... View more
01-24-2017
06:00 AM
@Kuldeep Kulkarni
I encountered similar issue with HDFS_CLIENT, I've successfully deleted the client using curl call as mentioned in your post.
But, Installation of HDFS_CLIENT stuck at:
2017-01-24 08:44:43,873 - Package['unzip'] {}
And installation failed with following error message:
stderr: /var/lib/ambari-agent/data/errors-2631.txt
Python script has been killed due to timeout after waiting 1800 secs Could you please suggest me some pointers?
... View more
10-04-2016
09:34 AM
After performing the mentioned steps i'm still facing the same issue. The alert is same: “Connection failed: [Errno 111] Connection refused to localhost:6667” curl -u admin:admin -H 'X-Requested-By: ambari' -X GET "http://localhost:8080/api/v1/clusters/XXXX/alert_definitions/17"
{
"href" : "http://localhost:8080/api/v1/clusters/flumetest/alert_definitions/17",
"AlertDefinition" : {
"cluster_name" : "XXXX",
"component_name" : "KAFKA_BROKER",
"description" : "This host-level alert is triggered if the Kafka Broker cannot be determined to be up.",
"enabled" : true,
"id" : 17,
"ignore_host" : false,
"interval" : 1,
"label" : "Kafka Broker Process",
"name" : "kafka_broker_process",
"scope" : "HOST",
"service_name" : "KAFKA",
"source" : {
"default_port" : 9092.0,
"reporting" : {
"critical" : {
"value" : 5.0,
"text" : "Connection failed: {0} to {1}:{2}"
},
"warning" : {
"value" : 1.5,
"text" : "TCP OK - {0:.3f}s response on port {1}"
},
"ok" : {
"text" : "TCP OK - {0:.3f}s response on port {1}"
}
},
"type" : "PORT",
"uri" : "{{kafka-broker/listeners}}"
}
}
}
After changing the listener port in config to : 9092 I'm getting: Connection failed: [Errno 111] Connection refused to localhost:9092
... View more