Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251

org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251

New Contributor

org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior

when I create parquet table and insert into a table into it using Spark

see PARQUET-251
Jun 25, 2018 4:50:24 PM INFO: org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251
Jun 25, 2018 4:50:24 PM INFO: org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251
Jun 25, 2018 4:50:24 PM INFO: org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251
Jun 25, 2018 4:50:24 PM INFO: org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251
Jun 25, 2018 4:50:24 PM INFO: org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251

1 REPLY 1
Highlighted

Re: org.apache.parquet.CorruptStatistics: Ignoring statistics because this file was created prior to 1.8.0, see PARQUET-251

@prashanth Venkataiah, Where are you seeing this warnings ? Is it hive or spark?

If in Hive Ref: https://community.hortonworks.com/content/supportkb/150730/warning-orgapacheparquetcorruptstatistics...

If it is in Spark : (Update the log4j by adding below line)

log4j.logger.org.apache.parquet=ERROR   
Don't have an account?
Coming from Hortonworks? Activate your account here