Temporary solution of blocking port 8088 is working for me as of now. ... View more

Temporary workaround could be set hadoop.http.staticuser.user=testuser assign testuser to queue testqueue with 1% resources ? ... View more

I am wondering if this a security loophole ,since my cluster is not yet kerberized ! ... View more

can you check if any cronjob triggering this ? crontab -l ! ... View more

Yeah I was thinking to change static user but this was not there until today afternoon. Its suddenly started spawning applications using dr.user. ... View more

I am also facing issue with dr.who user with MYYARN application submitting in loop. But those are staying in "ACCEPTED" status. Total 18 application are launched. no clue !! ... View more

Thanks for the reply !! Issue was resolved by increasing value for dfs.datanode.max.transfer.threads to 16000 in my case. Also increasing ulimit value on each worker node. Regards, Harshal ... View more

Same exception for orc hive table as well. looks like this is a generic issue for below case : 1. create external table T1(col A , B , C) with partition on (col A) stored as ORC . Load table with substantial data. in my case around 85 GB data. 2. Create external table T2(col A,B,C) with partition on (Col B) stored as ORC. Load table T2 from T1 with dynamic partition. Output :- Premature EOF exception Please try out ! ... View more

Thanks for your reply Jonas ! I have verified datanode health and its all fine, there are no corrupt blocks across filesystem. I will check by changing the format to ORC. ... View more

So you mean this is something to do with Parquet ? Parquet has good integration with Spark . ... View more