Hi, I am getting the error when try to run hive insert query with UDF. This is the error I get "Vertex failed, vertexName=Map 1, vertexId=vertex_1545806970486_0001_1_00, diagnostics=[Task failed, taskId=task_1545806970486_0001_1_00_000097, diagnostics=[TaskAttempt 0 failed, info=[Container container_e13_1545806970486_0001_01_000107 finished with diagnostics set to [Container completed. ]], TaskAttempt 1 killed, TaskAttempt 2 failed, info=[Container container_e13_1545806970486_0001_01_000136 received a STOP_REQUEST], TaskAttempt 3 failed, info=[Container container_e13_1545806970486_0001_01_000158 finished with diagnostics set to [Container completed. ]], TaskAttempt 4 failed, info=[Container container_e13_1545806970486_0001_01_000254 finished with diagnostics set to [Container completed. ]]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:230, Vertex vertex_1545806970486_0001_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE] Vertex killed, vertexName=Reducer 2, vertexId=vertex_1545806970486_0001_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:418, Vertex vertex_1545806970486_0001_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE] DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1545806970486_0001_1_00, diagnostics=[Task failed, taskId=task_1545806970486_0001_1_00_000097, diagnostics=[TaskAttempt 0 failed, info=[Container container_e13_1545806970486_0001_01_000107 finished with diagnostics set to [Container completed. ]], TaskAttempt 1 killed, TaskAttempt 2 failed, info=[Container container_e13_1545806970486_0001_01_000136 received a STOP_REQUEST], TaskAttempt 3 failed, info=[Container container_e13_1545806970486_0001_01_000158 finished with diagnostics set to [Container completed. ]], TaskAttempt 4 failed, info=[Container container_e13_1545806970486_0001_01_000254 finished with diagnostics set to [Container completed. ]]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:230, Vertex vertex_1545806970486_0001_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1545806970486_0001_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:418, Vertex vertex_1545806970486_0001_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1" "The error does not say much except for the vertex failed" I have already checked the other post which mentioned similar issue but it did not help. https://community.hortonworks.com/questions/48549/hive-vertex-issue.html https://community.hortonworks.com/questions/24730/hive-job-failed-on-tez.html https://community.hortonworks.com/questions/90648/hive-error-vertex-failed.html https://community.hortonworks.com/questions/140266/hive-query-error-with-vertex-failed-on-partitioned.html https://community.hortonworks.com/questions/141485/tez-vertex-failed-due-to-its-own-failuredag-did-no.html https://community.hortonworks.com/questions/222722/hive-query-fails-in-tez-runs-in-mr-mode.html I tried to set configuration as mentioned in one of the post set hive.execution.engine=tez; set hive.auto.convert.join=true; set hive.auto.convert.join.noconditionaltask=true; set hive.auto.convert.join.noconditionaltask.size=405306368; set hive.vectorized.execution.enabled=true; set hive.vectorized.execution.reduce.enabled =true; set hive.cbo.enable=true; set hive.compute.query.using.stats=true; set hive.stats.fetch.column.stats=true; set hive.stats.fetch.partition.stats=true; set hive.merge.mapfiles =true; set hive.merge.mapredfiles=true; set hive.merge.size.per.task=134217728; set hive.merge.smallfiles.avgsize=44739242; set mapreduce.job.reduce.slowstart.completedmaps=0.8; But this also did not work. My table structure is as follows: CREATE TABLE table_name(id string, record ARRAY<ARRAY<string>>) PARTITIONED BY (dateonly string, place string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '#' COLLECTION ITEMS TERMINATED BY ',' MAP KEYS TERMINATED BY '!' LINES TERMINATED BY '\n' STORED AS SEQUENCEFILE; My query structure is as follows: INSERT OVERWRITE TABLE table_name PARTITION (dateonly='some_date', place ='some_place') select id, UDF(id1,id2.......) as record from other_table where dateonly = 'some_date' and place = 'some_place' group by id; The thing is if I change execution engine to 'mr' it work but for 'tez' execution engine this error keeps coming. Please kindly help if anyone has any solution for this issue. Thanks in advance. ... View more

I am trying to delete some of the rows from my hive table which has partitions. This is what I did. delete from <table_name> where <condition>; However, I am getting following error. FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations. Please anyone suggest why the query is not working. Thanks in advance. ... View more

Hi I have Ambari Version 2.5.2.0 with 4 Nodes. 1 master and 3 slave. In HDFS config property I have set Block replication = 3 dfs.replication.max = 3 (Default value was 50) If I did not set dfs.replication.max to 3 I was getting some under replicated blocks. With dfs.replication.max = 3, there was no under replicated blocks. Now the Issue Part. I have another old hadoop system (hadoop 1.2.1) with 10 nodes. I want to copy the data from the old hadoop system to HDP using 'distcp'. But when I try to copy I keep on getting error as 'Requested replication 10 exceeds maximum 3'. Why is this error generated when try to use distcp. Please help solve this issue. Regards, Saurav Ranjit ... View more

Hi, I am trying to use ambari to for HDP. So far I have installed the ambari server successfully. however when i start the server I get the following error. " Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... No errors were found. ERROR: Exiting with exit code 1. REASON: Database check failed to complete. Please check /var/log/ambari-server/ambari-server.log and /var/log/ambari-server/ambari-server-check-database.log for more information. " OS: Centos 7 Java : Oracle java 1.7 Database: Postgressql 9.4 ambari-server.txt ambari-server-check-database.txt I have attached the log file of the server. Thank you for the help. ... View more