Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impalad daemon 2.3 crashing regularly

Impalad daemon 2.3 crashing regularly

Explorer

Hi,

We have had several daemon restarts and the last log entry before the restart was:

 

I1125 02:07:05.337430 32232 status.cc:114] File hdfs://nameservice1/bigdata/parquet/cmdb/cmdb_node_stats_1448445780889.parquet.snappy is invalid. Missing metadata.
@ 0x78b793 (unknown)
@ 0xbede4b (unknown)
@ 0xbee501 (unknown)
@ 0xbbb5aa (unknown)
@ 0xaa4faa (unknown)
@ 0xaa7130 (unknown)
@ 0xca79b3 (unknown)
@ 0x7f5ef0d3e9d1 start_thread
@ 0x7f5eefc9c9dd clone

 

 

Would this error be responsible for the restart? (Not too obvious looking at the code)

 

Would a refresh table cmd executed while the parquet table was being written and be incomplete cause this restart?

All indications points to this but would like a second opinion :)

Thanks in advance!

Stephane

1 REPLY 1
Highlighted

Re: Impalad daemon 2.3 crashing regularly

Contributor

Hi Stefanie,

 

Ideally, you don't want refresh statements to be interleaved with inserts. The error message indicates some stale metadata but we can't tell for sure the root cause of it. I would recommend using refresh statements as serialization points and see if the problem persists. 

 

Regards,

Dimitris