Member since
04-27-2016
218
Posts
133
Kudos Received
25
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3251 | 08-31-2017 03:34 PM | |
6791 | 02-08-2017 03:17 AM | |
2837 | 01-24-2017 03:37 AM | |
9899 | 01-19-2017 03:57 AM | |
5405 | 01-17-2017 09:51 PM |
08-31-2017
03:34 PM
It started working after I disabled the vectorized execution mode. set hive.vectorized.execution.enabled = false;
... View more
08-31-2017
03:04 PM
Thanks @Sindhu I tried that but got the same exception. The same query is working fine if I turn on the llap mode.
... View more
08-31-2017
12:47 AM
I am getting following exception while running one the TPCDS benchmarking query in non llap mode. Please advise. ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1504136552198_0002_33_05, diagnostics=[Task failed, taskId=task_1504136552198_0002_33_05_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async Initialization failed. abortRequested=false
at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:464)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:398)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:564)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:516)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:384)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:335)
... 15 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastLongHashTable.allocateBucketArray(VectorMapJoinFastLongHashTable.java:265)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastLongHashTable.<init>(VectorMapJoinFastLongHashTable.java:279)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastLongHashMap.<init>(VectorMapJoinFastLongHashMap.java:113)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.createHashTable(VectorMapJoinFastTableContainer.java:115)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.<init>(VectorMapJoinFastTableContainer.java:86)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:108)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:315)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:187)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:183)
at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:91)
at org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:108)
... 4 more
, errorMessage=Cannot recover from this error:java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async Initialization failed. abortRequested=false
at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:464)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:398)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:564)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:516)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:384)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:335)
... 15 more
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Tez
04-21-2017
02:27 AM
@Adda Fuentes Looks like similar issue , can you please check what nar file you have in <nifi_home>/lib folder? e.g. nifi-hive-nar-1.1.0.2.1.2.0-10.nar
... View more
04-04-2017
08:35 PM
@jwitt PutSQL : Insert into PCSOR (policy_num, prem_amount, LOB, AdminSystem, Name, DOB, Phone, Address) values ( '123456','2000.00','PC','PC SOR','John Smith','1/1/1970','212-010-2345','316 Lincoln Rd. Brooklyn, NY 11225') ON DUPLICATE KEY UPDATE Address='316 Lincoln Rd. Brooklyn, NY 11225'; Sample Flow: Connection Pool.
... View more
04-04-2017
08:21 PM
1 Kudo
I am getting following exception with PutSQL processor "2017-04-04 20:18:00,746 ERROR [Timer-Driven Process Thread-10] o.apache.nifi.processors.standard.PutSQL PutSQL[id=aff94ebf-739d-1936-ffff-ffff9e38bbdf] Failed to update database due to a failed batch update. There were a total of 1 FlowFiles that failed, 0 that succeeded, and 0 that were not execute and will be routed to retry;" The same sql if I run through command line works fine. There are no additional details for the exception.
... View more
Labels:
- Labels:
-
Apache NiFi