How access to Spark Web UI ?

[#1]

Hi,

I have a self-contained application with Spark 1.4.1, scala 2.11 and sbt 0.13.14. I run my application by sbt run then sbt package to genere a file jar.

I don't know where my application is runing, how much number of servers and into who cluster. I know that from Spark UI I can access to the interface, where I can see all my servers and I can change the sizes of a memory of the executor and driver. But the problem I don't know how I can access to a Spark web UI ?

I tried by 127.0.0.1:4040 but the page is inaccessible, when runing of my application it display that my driver in this address 10.0.2.15, I tried also by 10.0.2.15:4040 but in vein. Knowing that I'm using the Apache Ambari where I can access to all the clusters of hadoop ( I use Hadoop because my data is stored into HDFS).

Can you please tell me how I can access to the Saprk UI.

Thank you.

[#2]

@Alicia Alicia

if I understand correctly, you run Spark locally. Could you check what is running on your local port 4040?

I assume that you use a Linux-like kernel, including mac OS:

lsof -i :4040 | grep LISTEN

Write down the PID and use the following command to learn more

ps -ef WhatEverPID

If you have something else running on the port, then that's your issue. You have to stop and start Spark to use a different port for Web UI. Please make sure that is available.

It is unlikely that you have a firewall blocking port 4040, but that is another potential cause.

[#3]

Thak you for your answer. Yes I checked on availability of the port 4040 is reserved:

the result of checking is the following:

Active connexion local address Remote address state

TCP 127.0.0.1:4040 DESKTOP-FF4U.....:0 Listening

[#4]

@Alicia Alicia

What is on your 8080 port?

http://localhost:8080

Per https://spark.apache.org/docs/1.4.1/spark-standalone.html, spark master web ui is 8080

[#5]

@Constantin Stanca

Thank you for your answer, I'm sorry for this deplay in response.

In fact, the port 8080 is reserved for Ambari, http://localhost:8080, this is a print-screen:

[#6]

Alicia, please see my answer above on Oct 24. If you are running Spark on YARN you will have to go through the YARN RM UI to get to the Spark UI for a running job. Link for YARN UI is available from Ambari YARN service. For a completed job, you will need to go through Spark History Server. Link for Spark history server is available from the Ambari Spark service.