Spark2_Jobhistoryserver_Process

[#1]

Hello,

what is the purpose of Spark2_Jobhistoryserver_Process? . Under what circumstances we get alert for 'Spark2_Jobhistoryserver_Process' apart from stopping the service.

[#2 · Accepted solution]

@Nani Bigdata

The Spark2 History server allows us to review Spark application metrics after the application has completed. Without the History Server, the only way to obtain performance metrics is through the Spark UI while the application is running.

Regarding the alert "SPARK2_JOBHISTORYSERVER_PROCESS" it is basically a host-level alert which is triggered if the Spark2 History Server cannot be determined to be up. Basically this alert checks the "spark2-defaults/spark.history.ui.port" port accessibility to determine if the Spark2 Job History server is UP and Running or not ? If not then this alert will be triggered.

[#3]

@Nani Bigdata

Did that answer your query? It will be a good community practice to mark the answer as "Accept" if it answers your query or Post back if you have any additional query.

[#4]

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure

[#5]

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure