Hive query execution taking longer time

rdevprasad1 — Fri, 28 Jul 2017 00:02:35 GMT

hive query is running for longer time, with 2 mappers and 2 Reducers. One of the reducer is taking1000 tasks and other reducer has only 1 task. Can you explain why the tasks are not evenly split between reducers

Re: Hive query execution taking longer time

sreeviswa_athic — Fri, 28 Jul 2017 17:51:53 GMT

Time taking for Query execution depends on multiple factors

1. Mainly the Hive query design, joins and the columns being pulled

2. YARN/TEZ container size allocated, depends where you are running

3. Check the queue you are running your job, check if queue is free

to answer your question on why one of the reducer is taking 1000 tasks

please the hive.exec.reducers.max value defined

If you want to play and modify the number of reducers, try changing the value of hive.exec.reducers.bytes.per.reducer(preferably assign a smaller, as this value is inversely proportional to number of reducers)

question Hive query execution taking longer time in Archives of Support Questions (Read Only)

Hive query execution taking longer time

Re: Hive query execution taking longer time