Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark2_Jobhistoryserver_Process

avatar
Contributor

Hello,

what is the purpose of Spark2_Jobhistoryserver_Process? . Under what circumstances we get alert for 'Spark2_Jobhistoryserver_Process' apart from stopping the service.



1 ACCEPTED SOLUTION

avatar
Master Mentor

@Nani Bigdata

The Spark2 History server allows us to review Spark application metrics after the application has completed. Without the History Server, the only way to obtain performance metrics is through the Spark UI while the application is running.

Regarding the alert "SPARK2_JOBHISTORYSERVER_PROCESS" it is basically a host-level alert which is triggered if the Spark2 History Server cannot be determined to be up. Basically this alert checks the "spark2-defaults/spark.history.ui.port" port accessibility to determine if the Spark2 Job History server is UP and Running or not ? If not then this alert will be triggered.

View solution in original post

4 REPLIES 4

avatar
Master Mentor

@Nani Bigdata

The Spark2 History server allows us to review Spark application metrics after the application has completed. Without the History Server, the only way to obtain performance metrics is through the Spark UI while the application is running.

Regarding the alert "SPARK2_JOBHISTORYSERVER_PROCESS" it is basically a host-level alert which is triggered if the Spark2 History Server cannot be determined to be up. Basically this alert checks the "spark2-defaults/spark.history.ui.port" port accessibility to determine if the Spark2 Job History server is UP and Running or not ? If not then this alert will be triggered.

avatar
Master Mentor

@Nani Bigdata

Did that answer your query? It will be a good community practice to mark the answer as "Accept" if it answers your query or Post back if you have any additional query.

avatar
Contributor

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure

avatar
Contributor

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure