Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

LogFDsCacheFlushTimer thread increase

avatar
Explorer

i used HDP-2.3.4.0-3485.

I execute some queries per 150s vi hiveserver2

i found that the threads increase continue...

Of course, i closed session... but thread was not kill..

so i checked the hiveserver2 using jstack

i found

Full thread dump

 Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode):
"IPC Client (1445384588) connection to lognodem.hadoop.com/10.10.82.223:8020 from anonymous" #7175 daemon prio=5 os_prio=0 tid=0x00007f9bc546d800 nid=0x447b in Object.wait() [0x00007f9b8d494000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:931)
        - locked <0x0000000745d59d28> (a org.apache.hadoop.ipc.Client$Connection)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:976)
"IPC Parameter Sending Thread #178" #7173 daemon prio=5 os_prio=0 tid=0x00007f9bc4edc800 nid=0x3b1b waiting on condition [0x00007f9b89959000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x000000064b8bf9f0> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
"LogFDsCachecleanInActiveFDsTimer" #6810 daemon prio=5 os_prio=0 tid=0x00007f9bc41cc800 nid=0x1c0 in Object.wait() [0x00007f9b87e40000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075726d230> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6809 daemon prio=5 os_prio=0 tid=0x00007f9bc4227800 nid=0x1bf in Object.wait() [0x00007f9b8a161000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075726d548> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6800 daemon prio=5 os_prio=0 tid=0x00007f9bc45a0800 nid=0x1b3 in Object.wait() [0x00007f9b88446000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075726df98> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6799 daemon prio=5 os_prio=0 tid=0x00007f9bc49a9000 nid=0x1b2 in Object.wait() [0x00007f9b8c787000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075726e2b0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6788 daemon prio=5 os_prio=0 tid=0x00007f9bc459e000 nid=0x177 in Object.wait() [0x00007f9b907c7000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000757286c88> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6787 daemon prio=5 os_prio=0 tid=0x00007f9bc422a800 nid=0x176 in Object.wait() [0x00007f9b89757000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000757286fa0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6778 daemon prio=5 os_prio=0 tid=0x00007f9bc4834000 nid=0x16b in Object.wait() [0x00007f9b88749000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000007572141a0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6777 daemon prio=5 os_prio=0 tid=0x00007f9bc4171800 nid=0x16a in Object.wait() [0x00007f9b8e0a0000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000757213420> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6765 daemon prio=5 os_prio=0 tid=0x00007f9bc546b000 nid=0x153 in Object.wait() [0x00007f9b8894b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000757288190> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6764 daemon prio=5 os_prio=0 tid=0x00007f9bc4c2f800 nid=0x152 in Object.wait() [0x00007f9b88a4c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000007572884a8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6755 daemon prio=5 os_prio=0 tid=0x00007f9bc422b000 nid=0x147 in Object.wait() [0x00007f9b8ab6b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000757288ef8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6754 daemon prio=5 os_prio=0 tid=0x00007f9bc429d000 nid=0x146 in Object.wait() [0x00007f9b97a37000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000757289210> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6744 daemon prio=5 os_prio=0 tid=0x00007f9bc5175800 nid=0x13d in Object.wait() [0x00007f9b89f5f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728a268> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6743 daemon prio=5 os_prio=0 tid=0x00007f9bc4b19000 nid=0x13c in Object.wait() [0x00007f9b9b370000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728a580> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6739 daemon prio=5 os_prio=0 tid=0x00007f9bc5173800 nid=0x138 in Object.wait() [0x00007f9b8a666000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728b550> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6738 daemon prio=5 os_prio=0 tid=0x00007f9bc4c42800 nid=0x137 in Object.wait() [0x00007f9b91ada000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728b868> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6734 daemon prio=5 os_prio=0 tid=0x00007f9bc5153800 nid=0x133 in Object.wait() [0x00007f9b89151000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728c838> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6733 daemon prio=5 os_prio=0 tid=0x00007f9bc449c000 nid=0x132 in Object.wait() [0x00007f9b89252000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728cb50> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6727 daemon prio=5 os_prio=0 tid=0x00007f9bc449a000 nid=0x7ffe in Object.wait() [0x00007f9b8a767000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728d3f8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6726 daemon prio=5 os_prio=0 tid=0x00007f9bc4d10000 nid=0x7ffd in Object.wait() [0x00007f9b8ca8a000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000075728d710> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6717 daemon prio=5 os_prio=0 tid=0x00007f9bc4496000 nid=0x7fc2 in Object.wait() [0x00007f9b89555000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651f9dbe0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6716 daemon prio=5 os_prio=0 tid=0x00007f9bc4172000 nid=0x7fc1 in Object.wait() [0x00007f9b8a868000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651f9def8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6704 daemon prio=5 os_prio=0 tid=0x00007f9bc4281800 nid=0x7fb1 in Object.wait() [0x00007f9b8eaaa000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651f66878> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6703 daemon prio=5 os_prio=0 tid=0x00007f9bc413c000 nid=0x7fb0 in Object.wait() [0x00007f9b8be7e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651f5a4a8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6694 daemon prio=5 os_prio=0 tid=0x00007f9bc496b800 nid=0x7f9b in Object.wait() [0x00007f9b89858000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651f59b28> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6693 daemon prio=5 os_prio=0 tid=0x00007f9bc44d5000 nid=0x7f9a in Object.wait() [0x00007f9b8ac6c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651f59810> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6680 daemon prio=5 os_prio=0 tid=0x00007f9bc4aee800 nid=0x7d41 in Object.wait() [0x00007f9b8c383000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651cc2d80> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6679 daemon prio=5 os_prio=0 tid=0x00007f9bc477d800 nid=0x7d40 in Object.wait() [0x00007f9b8c484000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651cc2828> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6670 daemon prio=5 os_prio=0 tid=0x00007f9bc4296000 nid=0x7d36 in Object.wait() [0x00007f9b89c5c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651519540> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6669 daemon prio=5 os_prio=0 tid=0x00007f9bc49a2000 nid=0x7d35 in Object.wait() [0x00007f9b89d5d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651546b68> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6658 daemon prio=5 os_prio=0 tid=0x00007f9bc465d800 nid=0x7d29 in Object.wait() [0x00007f9b8af6f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651b3ede0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6657 daemon prio=5 os_prio=0 tid=0x00007f9bc4150000 nid=0x7d28 in Object.wait() [0x00007f9b8bd7d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651b3f0f8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6648 daemon prio=5 os_prio=0 tid=0x00007f9bc4cd9800 nid=0x7ced in Object.wait() [0x00007f9b92cec000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000065157fba0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6647 daemon prio=5 os_prio=0 tid=0x00007f9bc4157000 nid=0x7cec in Object.wait() [0x00007f9b8b171000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650e40660> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6635 daemon prio=5 os_prio=0 tid=0x00007f9bc4c37800 nid=0x7cda in Object.wait() [0x00007f9b8a262000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650edd280> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6634 daemon prio=5 os_prio=0 tid=0x00007f9bc49a5800 nid=0x7cd9 in Object.wait() [0x00007f9b8a363000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650edd598> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6625 daemon prio=5 os_prio=0 tid=0x00007f9bc413a800 nid=0x7cca in Object.wait() [0x00007f9b8b878000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f10c88> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6624 daemon prio=5 os_prio=0 tid=0x00007f9bc472d800 nid=0x7cc9 in Object.wait() [0x00007f9b8d393000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f21828> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6613 daemon prio=5 os_prio=0 tid=0x00007f9bc4d13800 nid=0x7cbd in Object.wait() [0x00007f9b8e1a1000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f5e5a0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6612 daemon prio=5 os_prio=0 tid=0x00007f9bc4dfe000 nid=0x7cbc in Object.wait() [0x00007f9b8bf7f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f23030> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6608 daemon prio=5 os_prio=0 tid=0x00007f9bc4667000 nid=0x7cb7 in Object.wait() [0x00007f9b8a969000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000065192ebc8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6607 daemon prio=5 os_prio=0 tid=0x00007f9bc4666800 nid=0x7cb6 in Object.wait() [0x00007f9b8aa6a000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000065192eee0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6598 daemon prio=5 os_prio=0 tid=0x00007f9bc429a000 nid=0x7cac in Object.wait() [0x00007f9b8ba7a000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006519153a0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6597 daemon prio=5 os_prio=0 tid=0x00007f9bc4295000 nid=0x7cab in Object.wait() [0x00007f9b91ede000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000065191af28> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6585 daemon prio=5 os_prio=0 tid=0x00007f9bc4148000 nid=0x7c9d in Object.wait() [0x00007f9b8ad6d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f652f0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6584 daemon prio=5 os_prio=0 tid=0x00007f9bc4afb800 nid=0x7c9c in Object.wait() [0x00007f9b8ae6e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f71978> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6575 daemon prio=5 os_prio=0 tid=0x00007f9bc427f800 nid=0x7c91 in Object.wait() [0x00007f9b8c585000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x000000065186c610> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6574 daemon prio=5 os_prio=0 tid=0x00007f9bc4157800 nid=0x7c90 in Object.wait() [0x00007f9b8d191000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509b0950> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6563 daemon prio=5 os_prio=0 tid=0x00007f9bc4155000 nid=0x7c55 in Object.wait() [0x00007f9b90ece000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650a81930> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6562 daemon prio=5 os_prio=0 tid=0x00007f9bc4524800 nid=0x7c54 in Object.wait() [0x00007f9b8d898000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650a81a38> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6558 daemon prio=5 os_prio=0 tid=0x00007f9bc4d11800 nid=0x7c4e in Object.wait() [0x00007f9b926e6000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006514872a8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6557 daemon prio=5 os_prio=0 tid=0x00007f9bc43ec000 nid=0x7c4d in Object.wait() [0x00007f9b8cb8b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000651486d68> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6548 daemon prio=5 os_prio=0 tid=0x00007f9bc4229000 nid=0x7c37 in Object.wait() [0x00007f9b8b575000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650a82590> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6547 daemon prio=5 os_prio=0 tid=0x00007f9bc4228800 nid=0x7c36 in Object.wait() [0x00007f9b8b676000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650aa7030> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6534 daemon prio=5 os_prio=0 tid=0x00007f9bc495d000 nid=0x79e9 in Object.wait() [0x00007f9b8b979000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650aa7cf8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6533 daemon prio=5 os_prio=0 tid=0x00007f9bc472a800 nid=0x79e8 in Object.wait() [0x00007f9b8ce8e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650aa7e00> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6524 daemon prio=5 os_prio=0 tid=0x00007f9bc4141000 nid=0x79dd in Object.wait() [0x00007f9b8f1b1000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006513f4768> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6523 daemon prio=5 os_prio=0 tid=0x00007f9bc54b3800 nid=0x79dc in Object.wait() [0x00007f9b904c4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006513fe7b8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6507 daemon prio=5 os_prio=0 tid=0x00007f9bc4aff800 nid=0x799a in Object.wait() [0x00007f9b8bb7b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f71e90> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6506 daemon prio=5 os_prio=0 tid=0x00007f9bc429b000 nid=0x7999 in Object.wait() [0x00007f9b8bc7c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f8ca40> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6497 daemon prio=5 os_prio=0 tid=0x00007f9bc5167800 nid=0x798e in Object.wait() [0x00007f9b8c181000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f8cf38> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6496 daemon prio=5 os_prio=0 tid=0x00007f9bc5167000 nid=0x798d in Object.wait() [0x00007f9b8c282000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x0000000650f72570> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6485 daemon prio=5 os_prio=0 tid=0x00007f9bc5360800 nid=0x797b in Object.wait() [0x00007f9b8e8a8000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509b1f68> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6484 daemon prio=5 os_prio=0 tid=0x00007f9bc4c45800 nid=0x797a in Object.wait() [0x00007f9b8febe000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509b1d80> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6475 daemon prio=5 os_prio=0 tid=0x00007f9bc5531000 nid=0x796f in Object.wait() [0x00007f9b8fdbd000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509ec3c8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6474 daemon prio=5 os_prio=0 tid=0x00007f9bc4528800 nid=0x796e in Object.wait() [0x00007f9b8f2b2000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509ec560> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6463 daemon prio=5 os_prio=0 tid=0x00007f9bc44c5800 nid=0x7963 in Object.wait() [0x00007f9b922e2000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509ddb10> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6462 daemon prio=5 os_prio=0 tid=0x00007f9bc431d800 nid=0x7962 in Object.wait() [0x00007f9b8dd9d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509b2700> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6457 daemon prio=5 os_prio=0 tid=0x00007f9bc4159000 nid=0x7956 in Object.wait() [0x00007f9b8c888000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509a6980> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6456 daemon prio=5 os_prio=0 tid=0x00007f9bc4c3f800 nid=0x7955 in Object.wait() [0x00007f9b8c989000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509a6c98> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6447 daemon prio=5 os_prio=0 tid=0x00007f9bc43ea800 nid=0x7948 in Object.wait() [0x00007f9b8cc8c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006506dc628> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6446 daemon prio=5 os_prio=0 tid=0x00007f9bc42c6000 nid=0x7947 in Object.wait() [0x00007f9b8cd8d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509a8c10> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6435 daemon prio=5 os_prio=0 tid=0x00007f9bc52e7000 nid=0x793c in Object.wait() [0x00007f9b8d999000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509a7d68> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6434 daemon prio=5 os_prio=0 tid=0x00007f9bc4755000 nid=0x793b in Object.wait() [0x00007f9b94908000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006509a8080> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6429 daemon prio=5 os_prio=0 tid=0x00007f9bc42ca000 nid=0x7904 in Object.wait() [0x00007f9b8cf8f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006506c6ec8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6428 daemon prio=5 os_prio=0 tid=0x00007f9bc42c9800 nid=0x7903 in Object.wait() [0x00007f9b8d090000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006506c71e0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCachecleanInActiveFDsTimer" #6419 daemon prio=5 os_prio=0 tid=0x00007f9bc472c800 nid=0x78f6 in Object.wait() [0x00007f9b8d595000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006506c7b90> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
"LogFDsCacheFlushTimer" #6418 daemon prio=5 os_prio=0 tid=0x00007f9bc472b800 nid=0x78f5 in Object.wait() [0x00007f9b8d696000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000006506cf0c8> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)
LogFDsCachecleanInActiveFDsTimer was increase....
LogFDsCacheFlushTimer was increaase...
so my process was died in few days..
What can i do for slove this problem..
sorry my english is not good..
thank you
1 ACCEPTED SOLUTION

avatar
Master Mentor
@SangHun Baek

This may be related to this https://issues.apache.org/jira/browse/TEZ-3017

https://issues.apache.org/jira/browse/HIVE-12766

This is fixed in 2.4

Workaround is : Disable ATS.

YARN => Configs => Advanced.

Then search "yarn.timeline-service.enabled", then please uncheck this. After saving changes Ambari will as you to restart related services, then please restart those. Once restarting has been completed, please feel free to stop "App Timeline Server"

2453-screen-shot-2016-02-29-at-14755-pm.png

View solution in original post

4 REPLIES 4

avatar
Master Mentor
@SangHun Baek

This may be related to this https://issues.apache.org/jira/browse/TEZ-3017

https://issues.apache.org/jira/browse/HIVE-12766

This is fixed in 2.4

Workaround is : Disable ATS.

YARN => Configs => Advanced.

Then search "yarn.timeline-service.enabled", then please uncheck this. After saving changes Ambari will as you to restart related services, then please restart those. Once restarting has been completed, please feel free to stop "App Timeline Server"

2453-screen-shot-2016-02-29-at-14755-pm.png

avatar
Explorer

Thank you~^^

avatar
Master Mentor

@SangHun Baek Please accept the answer to close the thread

avatar
New Contributor

Awesome, i've been stuck debugging this problem for 2 weeks now.

Thanks!