Skip to main content
LLL_
LLL_
Bronze ⭐️⭐️⭐️
October 10, 2023
Question

Where does the Hadoop WebUI data come from?

 

Hadoop commands take a while, usually a few seconds, to response.

While the information on the webUI is presented with almost no delay.

Does the information come from some APIs, logs or other?

I'd like to know the data source of WebUI and how could I get it.

Thanks a lot.

LLL__0-1696905438150.png

 

2 replies

rki_
Gold ⭐️
rki_
Gold ⭐️
October 10, 2023

@LLL_ Both the information be it either from hadoop cmdline or from Web UI will come from the in-memory data structures that the Namenode stores in its metadata. The NameNode maintains this metadata in its memory for fast access.

So the fsimage is your persistent metadata that the Namenode reads while starting up and keeps this information in its memory to present a dynamic state of the file system when the user queries via Web UI.

Translate
Meepoljd
Meepoljd
Gold ⭐️
January 5, 2024

I think your intention is to retrieve these data for your own monitoring or reporting tasks. If so, you can try requesting JMX to obtain the relevant data, such as through http://namenode:port/jmx.

Translate
Terms & ConditionsAccessibility statement