Why are there dr.who "MYYARN" applications running and all failing in what seems to be a loop?

[#1]

We're running a HDP 2.5 cluster and today we noticed a series of dr.who "MYYARN" applications running, failing, and then resubmitting to YARN again and again. In what seems to be an "infinite loop". We can't figure out what the applications are doing and why they are failing. Any thoughts? Many thanks in advance!

[#2]

I'm hitting exactly the same issue here with HDP 2.6.

It looks like some kind of DOS attack but I have no clue on how to handle this?

Any help from Hortonworks would be appreciated.

[#3]

I am also facing issue with dr.who user with MYYARN application submitting in loop. But those are staying in "ACCEPTED" status. Total 18 application are launched. no clue !!

[#4]

Mine are also staying in "ACCEPTED" status. And they're launched every 3 seconds... It's becoming a problem on the ResourceManager. When I look at the logs, I can only see actions coming from within my cluster.

[#5]

did you check this:
https://community.hortonworks.com/questions/2349/tip-when-you-get-a-message-in-job-log-user-dr-who.h...

The solution there was:
RESOLUTION Customer changed the following property in core-site.xml to resolve the issue. Other values such as hdfs or mapred also resolve the issue. If the cluster is managed by Ambari, this should be added in Ambari > HDFS > Configurations>Advanced core-site > Add Property

hadoop.http.staticuser.user=yarn

[#6]

Yeah I was thinking to change static user but this was not there until today afternoon. Its suddenly started spawning applications using dr.user.

[#7]

I did the change but it didn't change anything. Instead of being 'dr.who', it's now 'yarn' user that is feeding applications every 3 seconds that get stuck as "ACCEPTED". I still can't find how these applications are being triggered. Any other clue?

[#8]

can you check if any cronjob triggering this ? crontab -l !

[#9]

Thanks but I did check beforehand and there was no crontab whatsoever running for any user on any machine in my cluster.

[#10]

and no user connected to the system to start jobs in your cluster?