Wrong spark history redirection for finished jons

[mbigelow Champion]

Try setting the spark.yarn.historyServer.address manually as I mentioned previously.

[mbigelow Champion]

It is a Spark configuration setting.

"Otherwise you will need to specify it manually in the Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-defaults.conf or History Server Advanced Configuration Snippet (Safety Valve) for spark-history-server.conf."

[Fawze]

Tried both with no success

Under /etc/spark/conf.cloudera.spark_on_yarn/spark-defaults.conf at the 2 RM i added also tried to add it using CM and still not directing to the right history server.

spark.yarn.historyServer.address=http://avvr-ahc101.lpdomain.com:18088

[mbigelow Champion]

When you added it through CM did it show up in the spark-default.conf?

[Fawze]

No, In one RM it was but the other RM was not, i changed it manually in the spark default at the 2 RMs and kept if the spark client configuration through CM

[Fawze]

i searched all the config files at the resource managers for spark and all of them directs the history spark server to the right one, where i can track the error of the redirection.

[Fawze]

Any new insights?

[Fawze]

Anyone who faced the same issue or have any insight how to solve this issue?

[neerjakhattar]

This issue only can occur if spark-default.conf has the wrong config and getting overwritten from safety valve or it got changed manually from command line.

Maybe try deleting the role and add on a new host and do deploy client config and see what happens.

[Fawze]

This issue only can occur if spark-default.conf has the wrong config and getting overwritten from safety valve or it got changed manually from command line.

which servers i should check it's spark-default.conf about the this config, i checked both of resource managers and the parameters is right, and the safetly valve is empty at cloudera manager.

Maybe try deleting the role and add on a new host and do deploy client config and see what happens.

deleting the spark role means that i need to delete the oozie and hive before and i think it's not the right solution for this case.