hadoop shell command pretty slow

[#1]

favorite

I installed single-node hadoop cluster in my mac laptop. And today the hadoop shell (hadoop fs -ls ) become pretty slow. And it is pretty fast when I disable my wifi, but it is very slow when I enable my wifi. I don't know what's wrong with my wifi. I tried clean the dns cache, but doesn't work for me. Does anyone know what's wrong here ? Thanks

[#2]

Hi @jzhang,

DO you have another software on you laptop that connect to the web to download file or upload file? like Torrent or dropbox synchronization or big google drive synchronization?

I think when you connect to the web you start upload/downloading something and it use you disk iops => then VM goes slowly

Michel

[#3]

Hadoop is pretty fussy about networking and DNS, including, sometimes, reverse DNS. So you looking at the DNS cache could be going in the right direction. One thing I'd suspect is that its taking reverse DNS too long before it gives up, or there's some attempt being made to look up the local hostname every time some part of the system tries to connect to another.

Ideally, everything should be listening on and connecting to localhost, and there's an entry in /etc/hosts mapping localhost to 127.0.0.1. This isn't an Ubuntu Linux box is it, incidentally? They do very odd things with the host table.