02-28-2017 10:51 AM
02-28-2017 11:17 AM
03-01-2017 04:43 AM
Tried both with no success
Under /etc/spark/conf.cloudera.spark_on_yarn/spark-defaults.conf at the 2 RM i added also tried to add it using CM and still not directing to the right history server.
spark.yarn.historyServer.address=http://avvr-ahc101.lpdomain.com:18088
03-01-2017 10:04 AM
03-01-2017 10:07 AM
No, In one RM it was but the other RM was not, i changed it manually in the spark default at the 2 RMs and kept if the spark client configuration through CM
03-03-2017 04:17 PM
i searched all the config files at the resource managers for spark and all of them directs the history spark server to the right one, where i can track the error of the redirection.
03-05-2017 02:18 PM
Any new insights?
03-31-2017 11:23 AM
Anyone who faced the same issue or have any insight how to solve this issue?
04-03-2017 08:23 AM
This issue only can occur if spark-default.conf has the wrong config and getting overwritten from safety valve or it got changed manually from command line.
Maybe try deleting the role and add on a new host and do deploy client config and see what happens.
04-03-2017 08:29 AM
This issue only can occur if spark-default.conf has the wrong config and getting overwritten from safety valve or it got changed manually from command line.
which servers i should check it's spark-default.conf about the this config, i checked both of resource managers and the parameters is right, and the safetly valve is empty at cloudera manager.
Maybe try deleting the role and add on a new host and do deploy client config and see what happens.
deleting the spark role means that i need to delete the oozie and hive before and i think it's not the right solution for this case.