Reply
Expert Contributor
Posts: 203
Registered: ‎01-25-2017

Wrong spark history redirection for finished jons

Hi,

 I changed the spark history server in my cluster before several weeks.

I notice that when i'm clicking a finished spark job from cloudera manager, it's not directing me to the right spark history server url, while when i change the url manually i can see the finished the job.

Posts: 464
Topics: 1
Kudos: 58
Solutions: 38
Registered: ‎08-16-2016

Re: Wrong spark history redirection for finished jons

How did you move it? Did you remove the role form the one host and then add it to the other in CM?
Expert Contributor
Posts: 203
Registered: ‎01-25-2017

Re: Wrong spark history redirection for finished jons

Yes

Posts: 464
Topics: 1
Kudos: 58
Solutions: 38
Registered: ‎08-16-2016

Re: Wrong spark history redirection for finished jons

I haven't done Spark History server specific but not had issues with others. It could be a bug. I'll see if I can reproduce.

What CM/CDH versions?
Expert Contributor
Posts: 203
Registered: ‎01-25-2017

Re: Wrong spark history redirection for finished jons

CM 5.5.2

CDH 5.5.4

Expert Contributor
Posts: 203
Registered: ‎01-25-2017

Re: Wrong spark history redirection for finished jons

Just thinking to delete the spark service and then add it.

 

Anyone who experienced the same issue?

Posts: 464
Topics: 1
Kudos: 58
Solutions: 38
Registered: ‎08-16-2016

Re: Wrong spark history redirection for finished jons

What value is spark.yarn.historyServer.address set to? I suspect that it did not get updated to the new Spark History server address so the RM UI is still linking to the old location? This setting is not in CM. I had to check the spark configs on a node.

To try to fix it you could de-select YARN as the YARN (MR2) Service in CM by clicking on the blue arrow pointing to the left. Save the configs and then re-select YARN. Otherwise you will need to specify it manually in the Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-defaults.conf or History Server Advanced Configuration Snippet (Safety Valve) for spark-history-server.conf.
Expert Contributor
Posts: 203
Registered: ‎01-25-2017

Re: Wrong spark history redirection for finished jons

[ Edited ]
YARN (MR2 Included) Service
 
Spark (Service-Wide)
 
YARN (MR2 Included)
 
  • Missing required value: YARN (MR2 Included) Service

 

Cann't find spark.yarn.historyServer.address at any DataNode /etc/spark/conf files

Posts: 464
Topics: 1
Kudos: 58
Solutions: 38
Registered: ‎08-16-2016

Re: Wrong spark history redirection for finished jons

There shouldn't be any Spark components installed on the DN. Better stated, if there isn't a Spark role installed on a node it will not have the configs from CM. I have mine installed on the master and edge nodes and can see the configs there including the spark.yarn.historyServer.address setting.

It will complain that it is missing a requirement when you select it and that will go away after you re-select it.
Expert Contributor
Posts: 203
Registered: ‎01-25-2017

Re: Wrong spark history redirection for finished jons

I have Spark Gateway on all DNs and the priephrals.

 

de selecting and re selecting the MR will have no affect and will not have any client configurations changes needed to deployed, also trying de selecting it ad run client configuration deploy will not work since it's alerting on the missing MR.

 

Is there any work around for this?

Announcements