I am writing to you with following case. Namely It looks like that task where reduce steps are necessary doesnt work on tez under HDP 2.6.5.
I base my question on simple example.
1. Connect to beeline through JDBC (!connect jdbc:hive2://<host>:<port>/default)
2. Hive.execution.engine=tez on default. When I am doing query like: Select count(*) from table everything works fine and gives results quickly.
3. The problem occurs when I want to perform actions with cluase as where. The query is hanging and nothing is happening with it. I even cannott see task on the YARN queue. Total vacuum ... Query is: Select count(*) from table where col1 = 'something'
4. Queries based just on the MAP works properly, so hence my conclusion that something is wrong with REDUCE at the level of processing through tez. Query: Select * from table where col1='something'
5. WhenI switch from hive.execution.engine=tez to hive.execution.engine=mr and perform query Select count(*) from table where col1 = 'something', the task is executed correctly.
In relation to my lengthy story, I would like you to ask, if do you have any ideas why things like that could happen or what to do to run query properly under tez engine.
Thank you very much for your help.
For such queries/agreggated queries hiveserver2 needs to start an application to get the results. Hence I would suggest you check from Yarn application RM UI if the application is started or not. This may be related to resource issues, do you know if the queue where the application is submitted has sufficient capacity?
Yes, I have checked that. Everything looks fine from RM UI perspective. This situation occurs even, if no other process is submitted, so definetelly something is wrong. Some time ago we turn on transaction isolation. Could it have any influence?