Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

(5.8.2-1.cdh5.8.2.p0.3) Hive on Spark "Completed" Job Stuck (YARN)

avatar
Contributor

Cloudera Manger 5.8.2

CDH 5.8.2-1.cdh5.8.2.p0.3

 

Hive on Spark Completed Job got stuck on YARN.

 

After the hive query finished and got the result, the spark process is still running on YARN (even though when we see the the spark process in ApplicationManager it's already in COMPLETED status).

Temp solution, we have to kill the job manually or else we can't submit another spark job because YARN things the resources is still being used.

 

What I did is to downgrade back my cluster to CDH 5.8.0

1 ACCEPTED SOLUTION

avatar
Rising Star
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar
Rising Star
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Rising Star
Could you please post an update so we can determine if there is a regression or not? Thanks

avatar
Contributor
I already downgraded my cluster to CDH 5.8.0 and I tried of what you said on the previous.

Yes you are correct, event though the job is completed, when I did HoS query and didn't quit the hive shell, it's still use YARN resources.

Thanks for pointing out the matter.

avatar
Rising Star
Awesome !! Thanks for the update.