question Re: Where does the Hadoop WebUI data come from? in Support Questions

Where does the Hadoop WebUI data come from?

LLL_ — Tue, 10 Oct 2023 02:45:51 GMT

Hadoop commands take a while, usually a few seconds, to response.

While the information on the webUI is presented with almost no delay.

Does the information come from some APIs, logs or other?

I'd like to know the data source of WebUI and how could I get it.

Thanks a lot.

Re: Where does the Hadoop WebUI data come from?

rki_ — Tue, 10 Oct 2023 07:53:42 GMT

@LLL_ Both the information be it either from hadoop cmdline or from Web UI will come from the in-memory data structures that the Namenode stores in its metadata. The NameNode maintains this metadata in its memory for fast access.

So the fsimage is your persistent metadata that the Namenode reads while starting up and keeps this information in its memory to present a dynamic state of the file system when the user queries via Web UI.

Re: Where does the Hadoop WebUI data come from?

Meepoljd — Fri, 05 Jan 2024 08:34:04 GMT

I think your intention is to retrieve these data for your own monitoring or reporting tasks. If so, you can try requesting JMX to obtain the relevant data, such as through http://namenode:port/jmx.