every time I execute a big hive query, my hadoop nodemanager killed by SEGSEGV. I'm using hadoop 2.8.0 and oracle jdk 1.8.0_131 The yarn java error file snippet as follow: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f4b766818b1, pid=10904, tid=0x00007f4b33fff700 # # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libc.so.6+0x1628b1] __strlen_sse2_pminub+0x11 # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # --------------- T H R E A D --------------- Current thread (0x00007f4b727d2800): JavaThread "ContainersLauncher #10" [_thread_in_native, id=16663, stack(0x00007f4b33eff000,0x00007f4b34000000)] siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x000000000000000f Registers: RAX=0x0000000000000000, RBX=0x00000000000000b3, RCX=0x000000000000000f, RDX=0x0000000000000006 RSP=0x00007f4b33ffbdf8, RBP=0x00007f4b381b7e70, RSI=0x00007f4b33ffc2d0, RDI=0x000000000000000f R8 =0x00007f4b33ffbe60, R9 =0x0000000000000008, R10=0x00000000fffff000, R11=0x00007f4b766963f4 R12=0x0000000000000022, R13=0x00007f4b381b7f10, R14=0x00007f4b77123840, R15=0x0000000000000004 RIP=0x00007f4b766818b1, EFLAGS=0x0000000000010283, CSGSFS=0x0000000000000033, ERR=0x0000000000000004 TRAPNO=0x000000000000000e Top of Stack: (sp=0x00007f4b33ffbdf8) 0x00007f4b33ffbdf8: 00007f4b76f1cbed 0000000000000000 0x00007f4b33ffbe08: 00007f4b33ffc6d0 00007f4b33ffc6d0 0x00007f4b33ffbe18: 00007f4b76f200e4 00007f4b76f200ed 0x00007f4b33ffbe28: 00007f4b33ffbe60 00000000016f0520 0x00007f4b33ffbe38: 00007f4b33ffbed0 0000000000000004 0x00007f4b33ffbe48: 00007f4b76f1bbe8 00007f4b33ffc6d0 0x00007f4b33ffbe58: 00007f4b33ffc2d0 00656e696c646d63 0x00007f4b33ffbe68: 0000000000000000 333030305f363938 0x00007f4b33ffbe78: 303030304bb40058 0000000000000000 0x00007f4b33ffbe88: 0000000000000000 0000000000000000 0x00007f4b33ffbe98: 0000000000000000 0000000000000000 0x00007f4b33ffbea8: 0000000000000000 0000000000000000 0x00007f4b33ffbeb8: 0000000000000000 0000000000000000 0x00007f4b33ffbec8: 0000000000000000 6d616e6500203a5d 0x00007f4b33ffbed8: 0000000000003a65 0000000000000000 0x00007f4b33ffbee8: 0000000000000000 0000000000000000 0x00007f4b33ffbef8: 0000000000000000 0000000000000000 0x00007f4b33ffbf08: 0000000000000000 0000000000000000 0x00007f4b33ffbf18: 0000000000000000 0000000000000000 0x00007f4b33ffbf28: 0000000000000000 0000000000000000 0x00007f4b33ffbf38: 0000000000000000 0000000000000000 0x00007f4b33ffbf48: 0000000000000000 0000000000000000 0x00007f4b33ffbf58: 0000000000000000 0000000000000000 0x00007f4b33ffbf68: 0000000000000000 0000000000000000 0x00007f4b33ffbf78: 0000000000000000 0000000000000000 0x00007f4b33ffbf88: 0000000000000000 0000000000000000 0x00007f4b33ffbf98: 0000000000000000 0000000000000000 0x00007f4b33ffbfa8: 0000000000000000 0000000000000000 0x00007f4b33ffbfb8: 0000000000000000 0000000000000000 0x00007f4b33ffbfc8: 0000000000000000 0000000000000000 0x00007f4b33ffbfd8: 0000000000000000 0000000000000000 0x00007f4b33ffbfe8: 0000000000000000 0000000000000000 Instructions: (pc=0x00007f4b766818b1) 0x00007f4b76681891: c0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 0x00007f4b766818a1: 31 c0 89 f9 83 e1 3f 66 0f ef c0 83 f9 30 77 1d 0x00007f4b766818b1: f3 0f 6f 0f 66 0f 74 c1 66 0f d7 d0 85 d2 0f 85 0x00007f4b766818c1: 4e 02 00 00 48 89 f8 48 83 e0 f0 eb 24 48 89 f8 Register to memory mapping: RAX=0x0000000000000000 is an unknown value RBX=0x00000000000000b3 is an unknown value RCX=0x000000000000000f is an unknown value RDX=0x0000000000000006 is an unknown value RSP=0x00007f4b33ffbdf8 is pointing into the stack for thread: 0x00007f4b727d2800 RBP=0x00007f4b381b7e70 is pointing into the stack for thread: 0x00007f4b73bd4800 RSI=0x00007f4b33ffc2d0 is pointing into the stack for thread: 0x00007f4b727d2800 RDI=0x000000000000000f is an unknown value R8 =0x00007f4b33ffbe60 is pointing into the stack for thread: 0x00007f4b727d2800 R9 =0x0000000000000008 is an unknown value R10=0x00000000fffff000 is an unknown value R11=0x00007f4b766963f4: <offset 0x1773f4> in /lib64/libc.so.6 at 0x00007f4b7651f000 R12=0x0000000000000022 is an unknown value R13=0x00007f4b381b7f10 is pointing into the stack for thread: 0x00007f4b73bd4800 R14=0x00007f4b77123840: snoopy_inputdatastorage_data+0 in /usr/local/snoopy/lib/libsnoopy.so at 0x00007f4b76f16000 R15=0x0000000000000004 is an unknown value Stack: [0x00007f4b33eff000,0x00007f4b34000000], sp=0x00007f4b33ffbdf8, free space=1011k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [libc.so.6+0x1628b1] __strlen_sse2_pminub+0x11 Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j java.lang.UNIXProcess.forkAndExec(I[B[B[BI[BI[B[IZ)I+0 j java.lang.UNIXProcess.<init>([B[BI[BI[B[IZ)V+30 j java.lang.ProcessImpl.start([Ljava/lang/String;Ljava/util/Map;Ljava/lang/String;[Ljava/lang/ProcessBuilder$Redirect;Z)Ljava/lang/Process;+433 j java.lang.ProcessBuilder.start()Ljava/lang/Process;+161 j org.apache.hadoop.util.Shell.runCommand()V+136 j org.apache.hadoop.util.Shell.run()V+23 j org.apache.hadoop.util.Shell$ShellCommandExecutor.execute()V+67 j org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(Lorg/apache/hadoop/yarn/server/nodemanager/executor/ContainerStartContext;)I+492 j org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call()Ljava/lang/Integer;+1166 j org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call()Ljava/lang/Object;+1 J 4490 C1 java.util.concurrent.FutureTask.run()V (126 bytes) @ 0x00007f4b5e50acdc [0x00007f4b5e50aa80+0x25c] j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95 j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub the completed log can be downloaded here yarn_hs_error.log it seems like a hadoop or jdk bug? ... View more