Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kernel BUG when activating Cgroup over YARN

Kernel BUG when activating Cgroup over YARN

Hi,

We activated cgroup using Ambari. When I run a hive job, I got 2 nodes of my cluster down and I saw some errors in the linux kernel log (see above).

HDP version: 2.3.4

Kernel version: 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Any idea please ?

Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.475150] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.475409] IP: [<ffffffff813700d1>] rb_next+0x1/0x50
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.475574] PGD 0 
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.475647] Oops: 0000 [#1] SMP 
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.475757] Modules linked in: udp_diag tcp_diag inet_diag x86_pkg_temp_thermal intel_powerclamp 8021q garp iomemory_vsl(POX) stp mrp llc coretemp kvm crct10dif_pclmul gpio_ich crc32_pclmul mei_me mei sb_edac joydev edac_core shpchp wmi dcdbas aesni_intel acpi_power_meter aes_x86_64 lrw gf128mul glue_helper ablk_helper lpc_ich cryptd ipmi_watchdog ipmi_poweroff mac_hid ipmi_devintf ipmi_si hid_generic usbhid hid igb ixgbe i2c_algo_bit dca ptp megaraid_sas pps_core mdio
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.477273] CPU: 12 PID: 0 Comm: swapper/12 Tainted: P           OX 3.13.0-96-generic #143-Ubuntu
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.477559] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.477811] task: ffff881003e81800 ti: ffff880803c0c000 task.ti: ffff880803c0c000
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.478057] RIP: 0010:[<ffffffff813700d1>]  [<ffffffff813700d1>] rb_next+0x1/0x50
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.478304] RSP: 0018:ffff880803c0de20  EFLAGS: 00010046
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.478479] RAX: 0000000000000000 RBX: ffff880036b3ec00 RCX: 0000000000000cd2
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.478712] RDX: 0000000002cc7842 RSI: ffff880036b3d000 RDI: 0000000000000010
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.478947] RBP: ffff880803c0de68 R08: ffff8807d4cabe00 R09: 0000000000000018
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.479181] R10: 0000000000000415 R11: 00000000000004f8 R12: 0000000000000000
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.479416] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000b71b00
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.479651] FS:  0000000000000000(0000) GS:ffff88080fac0000(0000) knlGS:0000000000000000
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.479916] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.480106] CR2: 0000000000000010 CR3: 0000000001c0e000 CR4: 00000000001407e0
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.480338] Stack:
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.480404]  ffff880803c0de68 ffffffff810a2cc2 000000000000d160 ffff88080fad3180
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.480661]  ffff881003e81c30 ffff88080fad3180 000000000000000c 0000000000000000
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.480913]  ffff880803c0dfd8 ffff880803c0dec8 ffffffff8172dd22 ffff881003e81800
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.481167] Call Trace:
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.481250]  [<ffffffff810a2cc2>] ? pick_next_task_fair+0x102/0x1b0
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.481453]  [<ffffffff8172dd22>] __schedule+0x142/0x7f0
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.481624]  [<ffffffff8172e909>] schedule_preempt_disabled+0x29/0x70
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.481830]  [<ffffffff810c1d88>] cpu_startup_entry+0x268/0x2b0
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.482015]  [<ffffffff8104278d>] start_secondary+0x21d/0x2d0
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.482192] Code: e5 48 85 c0 75 07 eb 19 66 90 48 89 d0 48 8b 50 10 48 85 d2 75 f4 48 8b 50 08 48 85 d2 75 eb 5d c3 31 c0 5d c3 0f 1f 44 00 00 55 <48> 8b 17 48 89 e5 48 39 d7 74 3b 48 8b 47 08 48 85 c0 75 0e eb 
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.483043] RIP  [<ffffffff813700d1>] rb_next+0x1/0x50
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.483204]  RSP <ffff880803c0de20>
Nov 14 09:44:10 node002.cassandra.hdp kernel: [318591.483318] CR2: 0000000000000010

Don't have an account?
Coming from Hortonworks? Activate your account here