Created on 12-02-2016 05:22 PM - edited 08-19-2019 01:56 AM
Hi,
I'm trying to do a select count(*) from tweets; using Hive CLI. But as you can see, it is taking to much time (i guess)
It is normal?
Any tip?
Regards
Created 12-02-2016 05:54 PM
The Tez job has not started: both the mapper and the reducer are in "pending" state and haven't yet been launched. Once launched they would enter "running" state.
Check yarn to ensure there is enough room in your queue to fit the containers (http://sandbox:8088). There isn't much ram in a sandbox, and it could all be taken up by a Spark instance or by the Tez instances of hiveserver2.
Created 12-02-2016 05:54 PM
The Tez job has not started: both the mapper and the reducer are in "pending" state and haven't yet been launched. Once launched they would enter "running" state.
Check yarn to ensure there is enough room in your queue to fit the containers (http://sandbox:8088). There isn't much ram in a sandbox, and it could all be taken up by a Spark instance or by the Tez instances of hiveserver2.
Created on 12-02-2016 06:09 PM - edited 08-19-2019 01:56 AM
Jean-Phillippe, as you may guess, i am new with the Sandbox. How can i know if there's enough room in my queue to fit containers?
These are my setting.
Created 12-06-2016 04:55 PM
Since you are in a sandbox, you need to reduce the amount of memory taken by each component so that they can fit (try running with 12 GB if you can). Reduce the memory footprint as follows:
YARN will need to fit at least one Tez AM (512 MB) and a couple Tez containers (512MB *2). You can check how much memory is allocated to YARN on the YARN config page "Memory allocated for all YARN containers on a node".
Created 12-07-2016 03:08 AM
Dead Jean-Phillipe.
I tried to follow your instructions. But, as you can see is giving me headaches
"YARN will need to fit at least one Tez AM (512 MB) and a couple Tez containers (512MB *2). You can check how much memory is allocated to YARN on the YARN config page "Memory allocated for all YARN containers on a node"."
Can you please guide me please.
Regards
Created 12-16-2016 12:56 AM
Hi Cristian, the amount of memory that yarn can allocate is controlled by the setting "Memory allocated for all YARN containers on a node" under YARN. Set this to 3 GB, and that should give enough room for Tez to run (it needs 2.5 GB if you follow the settings above.).
Created 12-21-2016 03:01 AM
I've placed more ram, added jar path missing and worked perfect.
Thanks for your help
Created 12-02-2016 06:22 PM
Job is not started at all. Please share more info for debugging(cluster size, query, data size, hive log, etc)
Created 12-02-2016 06:43 PM
Hi Zyang,
Can you tell me where can i find that information?
Regards
Created 12-02-2016 07:00 PM
Cluster size: yarn node -list
Hive log: /tmp/<username>/hive.log