Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark 2 jobs with OOZIE on HDP 2.6.5 hangs

Solved Go to solution
Highlighted

spark 2 jobs with OOZIE on HDP 2.6.5 hangs

New Contributor

Hello All,

I try to run a spark 2 job with oozie on HDP 2.6.5.

Oozie starts successful, but my spark 2 job keeps in running state.

on Resource manager UI, I got

Application Type: SPARK
Application Tags: oozie-7bf4f1cfb7c848de2db37230975b4450
Application Priority: 0 (Higher Integer value indicates higher priority)
YarnApplicationState: ACCEPTED: waiting for AM container to be allocated, launched and register with RM.


Kindly suggest.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

2 REPLIES 2

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

@chris herssens,

Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and see if your spark job goes to running state from accepted.

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

New Contributor

I had limited resource on the yarn queue that I used for my spark job.

Don't have an account?
Coming from Hortonworks? Activate your account here