Created 11-25-2015 09:23 AM
Hi,
We have had several daemon restarts and the last log entry before the restart was:
I1125 02:07:05.337430 32232 status.cc:114] File hdfs://nameservice1/bigdata/parquet/cmdb/cmdb_node_stats_1448445780889.parquet.snappy is invalid. Missing metadata.
@ 0x78b793 (unknown)
@ 0xbede4b (unknown)
@ 0xbee501 (unknown)
@ 0xbbb5aa (unknown)
@ 0xaa4faa (unknown)
@ 0xaa7130 (unknown)
@ 0xca79b3 (unknown)
@ 0x7f5ef0d3e9d1 start_thread
@ 0x7f5eefc9c9dd clone
Would this error be responsible for the restart? (Not too obvious looking at the code)
Would a refresh table cmd executed while the parquet table was being written and be incomplete cause this restart?
All indications points to this but would like a second opinion :)
Thanks in advance!
Stephane
Created 12-02-2015 05:56 PM
Hi Stefanie,
Ideally, you don't want refresh statements to be interleaved with inserts. The error message indicates some stale metadata but we can't tell for sure the root cause of it. I would recommend using refresh statements as serialization points and see if the problem persists.
Regards,
Dimitris