Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark2_Jobhistoryserver_Process

Solved Go to solution
Highlighted

Spark2_Jobhistoryserver_Process

New Contributor

Hello,

what is the purpose of Spark2_Jobhistoryserver_Process? . Under what circumstances we get alert for 'Spark2_Jobhistoryserver_Process' apart from stopping the service.



1 ACCEPTED SOLUTION

Accepted Solutions

Re: Spark2_Jobhistoryserver_Process

Super Mentor

@Nani Bigdata

The Spark2 History server allows us to review Spark application metrics after the application has completed. Without the History Server, the only way to obtain performance metrics is through the Spark UI while the application is running.

Regarding the alert "SPARK2_JOBHISTORYSERVER_PROCESS" it is basically a host-level alert which is triggered if the Spark2 History Server cannot be determined to be up. Basically this alert checks the "spark2-defaults/spark.history.ui.port" port accessibility to determine if the Spark2 Job History server is UP and Running or not ? If not then this alert will be triggered.

4 REPLIES 4

Re: Spark2_Jobhistoryserver_Process

Super Mentor

@Nani Bigdata

The Spark2 History server allows us to review Spark application metrics after the application has completed. Without the History Server, the only way to obtain performance metrics is through the Spark UI while the application is running.

Regarding the alert "SPARK2_JOBHISTORYSERVER_PROCESS" it is basically a host-level alert which is triggered if the Spark2 History Server cannot be determined to be up. Basically this alert checks the "spark2-defaults/spark.history.ui.port" port accessibility to determine if the Spark2 Job History server is UP and Running or not ? If not then this alert will be triggered.

Re: Spark2_Jobhistoryserver_Process

Super Mentor

@Nani Bigdata

Did that answer your query? It will be a good community practice to mark the answer as "Accept" if it answers your query or Post back if you have any additional query.

Re: Spark2_Jobhistoryserver_Process

New Contributor

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure

Re: Spark2_Jobhistoryserver_Process

New Contributor

Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure

Don't have an account?
Coming from Hortonworks? Activate your account here