Support Questions

Find answers, ask questions, and share your expertise

if one person’s query hangs, then the spark interpreter will be hung until it is restarted

 
1 REPLY 1

@Mustaquim Ulde

If your cluster is Kerborized and you run Zeppelin UI as anonymous, the jobs are running as a zeppelin principal. Saying this - there is only one SPARK job triggered in YARN. If the process hangs and the next one cannot start this looks like not enough resources allocated for YARN.

Share the following with me:

  • yarn.nodemanager.resource.memory-mb
  • yarn.scheudler.minimum-allocation-mb
  • yarn.nodemanager.resource.cpu-vcores

The physical size of RAM and no. of CPU on each of the worker nodes.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.