Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala Crashes on CPU with AVX-512

avatar
Rising Star

Hi all,

 

I added some new nodes to our impala cluster.

On the new nodes, Impala continues to crash with the following error:

 

F0521 12:34:46.838728  5699 llvm-codegen.cc:112] LLVM hit fatal error: Cannot select: 0x7f62888c84c0: ch = store<ST1[%loc.i]> 0x7f622ebc5850, 0x1068b980, 0x1068b850, undef:i64
  0x1068b980: i1,ch = load<LD1[%val_ptr.i.i27]> 0x7f626d3d1830, 0x7f622ebc54c0, undef:i64
    0x7f622ebc54c0: i64 = add 0x1068bd10, Constant:i64<8>
      0x1068bd10: i64,ch = CopyFromReg 0x7f626d3d1830, Register:i64 %vreg21
        0x7f622ebc5ab0: i64 = Register %vreg21
      0x7f62888c8d10: i64 = Constant<8>
    0x7f61b3668260: i64 = undef
  0x1068b850: i64,ch = CopyFromReg 0x7f626d3d1830, Register:i64 %vreg20
    0x7f62888c8850: i64 = Register %vreg20
  0x7f61b3668260: i64 = undef
In function: _ZN6impala26PartitionedAggregationNode12ProcessBatchILb0EEENS_6StatusEPNS_8RowBatchENS_13TPrefetchMode4typeEPNS_12HashTableCtxE.1

The issue seems to be related to an hardware incompatibility with CPUs that have AVX-512 instructions.

Indeed the problem is present only on the new machines and from impala logs I notices that impala uses avx-512 cpu flags for generating at runtime code to obtain fully-optimized functions.

 

 

The problem is solved in the new version of impala IMPALA-6291 but at the moment we cannot upgrade the cluster.

 

Do you know if there is the possibility to modify the compiler options used by impala ? 

 

Thanks

Alessio

 

 

 

 

 

1 ACCEPTED SOLUTION

avatar
Rising Star

Solved upgrading Impala to version 2.12

View solution in original post

1 REPLY 1

avatar
Rising Star

Solved upgrading Impala to version 2.12