Created on 07-29-2019 09:12 AM - edited 08-17-2019 04:29 PM
Hi Team,
When are running a hive query using Zeppelin notebook, zeppelin hangs immediately with screenshot as attached.
But, the same query is successful when executed via beeline on command line at the same time, so I think it should not be an issue from YARN or resources end.
Is this an issue from Zeppelin?
Could anyone help me on this, please.
Created 07-30-2019 08:40 AM
@Geoffrey Shelton Okot can you help me on this.
Created 07-30-2019 10:24 PM
What is the HDP and zeppelin version? Can you share your cluster configuration?
What values do you have for the below parameters?
yarn.nodemanager.resource.cpu-vcores
yarn.scheduler.minimum-allocation-vcores
yarn.scheduler.maximum-allocation-vcores
yarn.nodemanager.resource.cpu-vcores are by default ~80% of total vCPUs available on the machine.
HTH
Created 07-31-2019 04:56 AM
Thanks for your time and here is the answer:
HDP - 2.6.4.0
Zeppelin - 0.7.3
yarn.nodemanager.resource.cpu-vcores - 1
yarn.scheduler.minimum-allocation-vcores - 1
yarn.scheduler.maximum-allocation-vcores - 1
Cluster is of only 2 nodes and each NM has below configuration(s)
NM1:
[root@xxxx ~]# cat /proc/cpuinfo | awk '/^processor/{print $3}' | tail -1
3
[root@xxx ~]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 2
NM2:
[root@xxxx ~]# cat /proc/cpuinfo | awk '/^processor/{print $3}' | tail -1
1
[root@ ~]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
I am wondering, when the job is running with no issues on beeline, putty ; why it is not working on Zeppelin?
Is there any difference?