Hive is fetching different results, if we query on a specific partition its giving correct results, but when we do the query on a specific range its giving incorrect results. when we query on specific date range its fetching correct results date_dt (DATE'2018-03-31',DATE'2018-05-19',DATE'2018-04-28',DATE'2018-03-17') but when we do the query on specific range, its yielding incorrect results date_dt BETWEEN DATE'2017-07-08' AND DATE'2018-07-07' ... View more

screen-shot-2018-06-19-at-104245-pm.png Count distinct doesn't always give me the right answer. I've attached two different queries that should both result in 7 unique items purchased. If I don't do an operation on mdse_item_i like cast it to a bigint, it doesn't always count them correctly. to explain it simply, when i do cast on "mdse_itcountsdistinct-1.txtdoesnotcountdistinct.txtem_i" it gives unique results as 7, but when i don't do casting it gives unique results as 10 which is not correct. hive> select * from dfr_distinct; OK 100000000938 5 7 12.33 2 2.75 4.27 8.060 2 8 0 Time taken: 0.479 seconds, Fetched: 1 row(s) hive> select * from dfr_distinctnot; OK 100000000938 5 10 12.33 2 2.75 4.27 8.06 0 2 8 0 Time taken: 0.932 seconds, Fetched: 1 row(s) tried running the query in both MR and Tez modes still giving same results when i don't do casting. ... View more

Hello HCC, We have 16 node cluster running on HDP 2.5.3.0, Ambari 2.5.1.0, on this cluster we allocated resource manager heap to 4G and when there is no single job running in the cluster for almost 5-6hrs, but still resource manager heap usage is going to 80-85% always. We have another big production cluster with 800 data nodes in that cluster RM usage will be under 25-30% always.On what basis heap usage is utilized on the cluster ? and how can we control heap usage when cluster is idle. i also checked this property "yarn.resourcemanager.max-completed-applications" which is hard coded to "${yarn.resourcemanager.max-completed-applications}" but this hard coded property not defined anywhere in Ambari. Thanks in advance HCC. ... View more

Hello HCC, We recently upgraded our prod and all dev cluster from HDP 2.5.3.0 to HDP 2.6.1.0, as we are observing weird behavior in HDP 2.6.1.0 some of the nodes are getting very high allocation of containers causing very high avg load on the server and that is causing nodes to go in heart beat lost state. When the nodes got very high avg load NN making those nodes as Dead nodes where as RM still keep on assigning containers ( we know that both RM and NN work independently) on that node and all those containers are causing jobs to go in failed state. Every time when we having this issue we are asking our SA team to reboot those servers to alleviate the issue, we didn't had this behavior with HDP 2.5.3.0. Please find the screenshot for reference where nodes got very high no.of containers and load avg Present versions : HDP 2.6.1.0 and Ambari 2.5.2.0 @Kuldeep Kulkarni @Jay SenSharma @Artem Ervits @ssathish ss-1.pngss-2.pngss-3.pngss-4.png ... View more

our jobs are failing with below exception we are using HDP 2.5.3.0 , Tez 0.7.0.2.5 , Hive 1.2.1.2.5 and Ambari 2.4.0 failing with below exception Status: Failed Vertex failed, vertexName=Reducer 2, vertexId=vertex_**, diagnostics=[Exception in VertexManager, vertex:vertex_ [Reducer 2],org.apache.tez.dag.api.TezUncheckedException: Atleast 1 bipartite source should exist at org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.onVertexStarted(ShuffleVertexManager.java:456) at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventOnVertexStarted.invoke(VertexManager.java:587) at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:658) at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:653) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:653) at org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) , Vertex did not succeed due to AM_USERCODE_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_** [Reducer 2] killed/failed due to:AM_USERCODE_FAILURE] Vertex did not succeed due to AM_USERCODE_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_ [Reducer 2] killed/failed due to:AM_USERCODE_FAILURE]Vertex killed, vertexName=Reducer 3, vertexId=vertex_, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_ [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Map 4, vertexId=vertex_, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_ [Map 4] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:2 After Staging execution is it something related to https://issues.apache.org/jira/browse/TEZ-3239 https://issues.apache.org/jira/browse/TEZ-2107 Thanks. ... View more