spark 2 jobs with OOZIE on HDP 2.6.5 hangs

[#1]

Hello All,

I try to run a spark 2 job with oozie on HDP 2.6.5.

Oozie starts successful, but my spark 2 job keeps in running state.

on Resource manager UI, I got

Application Type:	SPARK
Application Tags:	oozie-7bf4f1cfb7c848de2db37230975b4450
Application Priority:	0 (Higher Integer value indicates higher priority)
YarnApplicationState:	ACCEPTED: waiting for AM container to be allocated, launched and register with RM.

Kindly suggest.

[#2 · Accepted solution]

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

[#3]

I had limited resource on the yarn queue that I used for my spark job.