Support Questions
Find answers, ask questions, and share your expertise

spark 2 jobs with OOZIE on HDP 2.6.5 hangs

Solved Go to solution

spark 2 jobs with OOZIE on HDP 2.6.5 hangs

Hello All,

I try to run a spark 2 job with oozie on HDP 2.6.5.

Oozie starts successful, but my spark 2 job keeps in running state.

on Resource manager UI, I got

Application Type: SPARK
Application Tags: oozie-7bf4f1cfb7c848de2db37230975b4450
Application Priority: 0 (Higher Integer value indicates higher priority)
YarnApplicationState: ACCEPTED: waiting for AM container to be allocated, launched and register with RM.


Kindly suggest.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

View solution in original post

2 REPLIES 2

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

View solution in original post

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

I had limited resource on the yarn queue that I used for my spark job.