Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

spark 2 jobs with OOZIE on HDP 2.6.5 hangs

avatar
Contributor

Hello All,

I try to run a spark 2 job with oozie on HDP 2.6.5.

Oozie starts successful, but my spark 2 job keeps in running state.

on Resource manager UI, I got

Application Type: SPARK
Application Tags: oozie-7bf4f1cfb7c848de2db37230975b4450
Application Priority: 0 (Higher Integer value indicates higher priority)
YarnApplicationState: ACCEPTED: waiting for AM container to be allocated, launched and register with RM.


Kindly suggest.

1 ACCEPTED SOLUTION

avatar
Super Guru

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

View solution in original post

2 REPLIES 2

avatar
Super Guru

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

avatar
Contributor

I had limited resource on the yarn queue that I used for my spark job.