Definitely an attack.
I finally managed to spot a malicious IP address on my ResourceManager node that I could block.
We have the same issue, it started at the same moment (30 April and restarts 1 May) and we are using HDP 2.6, do you think is a Hortonworks error or a time bomb?
Please, if anyone finds a solution, please tell us in this post
This solution works, but it is strange that 5 different clusters of 5 different companies have the same attack at the same time.
I totally agree with this. It's weird. It sounds like some kind of time bomb. I tried blocking the 8088 port but it didn't work for me (at least not for long). MYYARN jobs kept on piling up. It's over 20,000 jobs for the last couple of days. Any help from Hortonworks team would be appreciated.
Is your cluster directly connected to the internet, so that any internet user can connect to your port 8088? And also your cluster is not kerberized?
There are regulary running kind of campaigns to search for unprotected or vulnerable services via Internet, so it shouldn't surprise that the attack is almost simultaneously hitting several clusters. There are even search engines available that will list you all services reachable from the internet, so that one can search for 'give me all unprotected hadoop machines'.
If your cluster is unprotected, the only solution will be to protect it, via firewall, via kerberos etc...
Can you guys check if you see the below process in your nodemanager machines?
/tmp/java -c /tmp/h.conf
according to ps, there is no process with "conf.h"; according to ls, there is no /tmp/java; checked on 2 nodes on 2 clusters
Temporary solution of blocking port 8088 is working for me as of now.
The problem is that the user dr.who who is launching the applications through the Ambari's API
We find another solution, we do not give access with ranger to the user dr.who and the port 8088 closed