Hi,
We've added a new node to a CDH 5.13.1 cluster and now the impala daemon of the new node is failling with IMPALAD_UNEXPECTED_EXITS. The following messages are shown in the log file:
F0517 12:14:30.510576 29320 llvm-codegen.cc:112] LLVM hit fatal error: Cannot select: 0x3aedbd10: ch = store<ST1[%sunkaddr29]> 0x47d56be0:1, 0x47d55e40, 0x1aace980, undef:i64
0x47d55e40: i1 = or 0x47d56be0, 0x47d56390
0x47d56be0: i1,ch = load<LD1[%sunkaddr29]> 0xbfd9830, 0x1aace980, undef:i64
0x1aace980: i64 = add 0x47d565f0, Constant:i64<32>
0x47d565f0: i64,ch = CopyFromReg 0xbfd9830, Register:i64 %vreg0
0x47d56260: i64 = Register %vreg0
0x47d574c0: i64 = Constant<32>
0x47d57130: i64 = undef
0x47d56390: i1 = truncate 0x47d555f0
0x47d555f0: i32 = srl 0x3aedb4c0, Constant:i8<8>
0x3aedb4c0: i32 = any_extend 0x47d55980
0x47d55980: i16,ch = CopyFromReg 0xbfd9830, Register:i16 %vreg8
0x480ae000: i16 = Register %vreg8
0x3aedbbe0: i8 = Constant<8>
0x1aace980: i64 = add 0x47d565f0, Constant:i64<32>
0x47d565f0: i64,ch = CopyFromReg 0xbfd9830, Register:i64 %vreg0
0x47d56260: i64 = Register %vreg0
0x47d574c0: i64 = Constant<32>
0x47d57130: i64 = undef
In function: _ZN6impala26PartitionedAggregationNode22ProcessBatchNoGroupingEPNS_8RowBatchE.12
*** Check failure stack trace: ***
@ 0x1b9c2bd google::LogMessage::Fail()
@ 0x1b9db62 google::LogMessage::SendToLog()
@ 0x1b9bc97 google::LogMessage::Flush()
@ 0x1b9f25e google::LogMessageFatal::~LogMessageFatal()
@ 0xc06f5a (unknown)
@ 0x1ae5553 llvm::report_fatal_error()
@ 0x1ae56fe llvm::report_fatal_error()
@ 0x11e03bb llvm::SelectionDAGISel::CannotYetSelect()
@ 0x11e15db llvm::SelectionDAGISel::SelectCodeCommon()
@ 0xffe787 (unknown)
@ 0x11dec04 llvm::SelectionDAGISel::DoInstructionSelection()
@ 0x11e49c2 llvm::SelectionDAGISel::CodeGenAndEmitDAG()
@ 0x11e871a llvm::SelectionDAGISel::SelectAllBasicBlocks()
@ 0x11e9a5f llvm::SelectionDAGISel::runOnMachineFunction()
@ 0x10027b4 (unknown)
@ 0x1a7ea8a llvm::FPPassManager::runOnFunction()
@ 0x1a7f053 llvm::legacy::PassManagerImpl::run()
@ 0x1733409 llvm::MCJIT::emitObject()
@ 0x17338d1 llvm::MCJIT::generateCodeForModule()
@ 0x1730440 llvm::MCJIT::finalizeObject()
@ 0xc0fc3d impala::LlvmCodeGen::FinalizeModule()
@ 0xa52bb6 impala::FragmentInstanceState::Open()
@ 0xa541cb impala::FragmentInstanceState::Exec()
@ 0xa30ad8 impala::QueryState::ExecFInstance()
@ 0xbd4812 impala::Thread::SuperviseThread()
@ 0xbd4f74 boost::detail::thread_data<>::run()
@ 0xe6122a (unknown)
@ 0x7f18339bde25 start_thread
@ 0x7f18336eb34d __clone
Picked up JAVA_TOOL_OPTIONS:
Wrote minidump to /var/log/impala-minidumps/impalad/ba97cae4-eb33-44ee-f569ebad-67095a51.dmp
We have stopped the new impala daemon in order to resolve the incident. The new node is different hardware than the current cluster nodes but all the configuration is deployed by Cloudera Manager 5.13.3.
Can you help us to determine the cause of the problem?
