Reply
Expert Contributor
Posts: 143
Registered: ‎08-07-2017

select count(*) inside hive giving error

Hello All,

 

I am trying to run select count(*) from table_name query in hive, but it is giving below error.

 

Diagnostic Messages for this Task:
Error: Java heap space

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

 

I tried using export HADOOP_CLIENT_OPTS=-Xmx4g and then launching hive , query executed once. but now its not working afterwards.

 

I used analyze table command also and inner query option as well. but no luck.

 

Please suggest.

 

Thanks,

Priya

Master
Posts: 430
Registered: ‎07-01-2015

Re: select count(*) inside hive giving error

Is it failing immediately or during the execution? Do you see the job running in YARN? If yes, try to increase map/reduce memories. If it fails immediately increase the client heap more - but it is wierd having 4g should be more than enough for a simple count..
Expert Contributor
Posts: 143
Registered: ‎08-07-2017

Re: select count(*) inside hive giving error

@Tomas79,

 

Thanks for inputs. Job is not failing immediately. After 81% completion of mappers, it is failing.Can you suggest how to increase map or reduce memory just for hive as our other applications are running fine.

 

Thanks,

Priya

Master
Posts: 430
Registered: ‎07-01-2015

Re: select count(*) inside hive giving error

For example for 4GB map and reduce memory set this via Hive

set mapreduce.map.memory.mb = 4096
set mapreduce.reduce.memory.mb = 4096