question Re: Spark2_Jobhistoryserver_Process in Support Questions

Spark2_Jobhistoryserver_Process

nanibigdata1 — Sun, 09 Jun 2019 00:04:00 GMT

Hello,

what is the purpose of Spark2_Jobhistoryserver_Process? . Under what circumstances we get alert for 'Spark2_Jobhistoryserver_Process' apart from stopping the service.

Re: Spark2_Jobhistoryserver_Process

jsensharma — Sun, 09 Jun 2019 05:33:46 GMT

@Nani Bigdata

The Spark2 History server allows us to review Spark application metrics after the application has completed. Without the History Server, the only way to obtain performance metrics is through the Spark UI while the application is running.

Regarding the alert "SPARK2_JOBHISTORYSERVER_PROCESS" it is basically a host-level alert which is triggered if the Spark2 History Server cannot be determined to be up. Basically this alert checks the "spark2-defaults/spark.history.ui.port" port accessibility to determine if the Spark2 Job History server is UP and Running or not ? If not then this alert will be triggered.

Re: Spark2_Jobhistoryserver_Process

jsensharma — Tue, 11 Jun 2019 11:12:37 GMT

@Nani Bigdata

Did that answer your query? It will be a good community practice to mark the answer as "Accept" if it answers your query or Post back if you have any additional query.

Re: Spark2_Jobhistoryserver_Process

nanibigdata1 — Fri, 14 Jun 2019 01:25:33 GMT

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure

Re: Spark2_Jobhistoryserver_Process

nanibigdata1 — Fri, 14 Jun 2019 01:25:46 GMT