Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Beeline JDBC connection - reduce does not work on tez

Beeline JDBC connection - reduce does not work on tez

New Contributor

Hi everyone,

I am writing to you with following case. Namely It looks like that task where reduce steps are necessary doesnt work on tez under HDP 2.6.5.

I base my question on simple example.

1. Connect to beeline through JDBC (!connect jdbc:hive2://<host>:<port>/default)

2. Hive.execution.engine=tez on default. When I am doing query like: Select count(*) from table everything works fine and gives results quickly.

3. The problem occurs when I want to perform actions with cluase as where. The query is hanging and nothing is happening with it. I even cannott see task on the YARN queue. Total vacuum ... Query is: Select count(*) from table where col1 = 'something'

4. Queries based just on the MAP works properly, so hence my conclusion that something is wrong with REDUCE at the level of processing through tez. Query: Select * from table where col1='something'

5. WhenI switch from hive.execution.engine=tez to hive.execution.engine=mr and perform query Select count(*) from table where col1 = 'something', the task is executed correctly.

In relation to my lengthy story, I would like you to ask, if do you have any ideas why things like that could happen or what to do to run query properly under tez engine.

Thank you very much for your help.

Best regards,

Hubert

2 REPLIES 2
Highlighted

Re: Beeline JDBC connection - reduce does not work on tez

@Hubert Poltorak

For such queries/agreggated queries hiveserver2 needs to start an application to get the results. Hence I would suggest you check from Yarn application RM UI if the application is started or not. This may be related to resource issues, do you know if the queue where the application is submitted has sufficient capacity?

HTH

Re: Beeline JDBC connection - reduce does not work on tez

New Contributor

Yes, I have checked that. Everything looks fine from RM UI perspective. This situation occurs even, if no other process is submitted, so definetelly something is wrong. Some time ago we turn on transaction isolation. Could it have any influence?