On the web portal: https://sparktesteus.azurehdinsight.net/yarnui/hn/cluster/apps/RUNNING , the listed running spark applications are wrong. The listed spark applications are actually finished. And the actual running applications are not present at the portal.
When I ssh to the head node of this Spark cluster, and use command 'yarn application -list', the command shows correct information. Why the web portal because stale.
Why the web portal is not reflecting the correct job id while the ssh shows the correct jobs id running?