Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Get Spark Job details

Get Spark Job details

Explorer

Hi,

I wanted to get all spark job details like Job ID, Job Name, Query, Start Time, End Time.  Is there an API to get these details. If there is one, do tell me please asap.

2 REPLIES 2

Re: Get Spark Job details

Rising Star

Hello @corestack,

 

Thanks for posting your query 

 

I am assuming you are talking about Spark-on-YARN (please correct me if I'm wrong), if so you may use Resource Manager APIs. 

 

https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster...

 

You can user Spark history server APIs too 

 

 

https://spark.apache.org/docs/2.2.0/monitoring.html#rest-api

Thanks,
Satz

Re: Get Spark Job details

Explorer

Hi Satz,

 

Thanks for your responce, I am not getting the full executed job details. It shows spark, But I need the executed query and its details.

 

Can you also tell me whether, we take HUE ACTIVE USERS count via API.

 

Thanks,

Don't have an account?
Coming from Hortonworks? Activate your account here