https://community.cloudera.com/t5/Support-Questions/Spark-1-6-1-how-to-skip-corrupted-parquet-blocks/m-p/103536#M66453 <P><A rel="user" href="https://community.cloudera.com/users/9304/tspann.html" nodeid="9304">@Timothy Spann</A> </P><P>Hi Timothy , </P><P>Thanks for the quick response. So, parquet file's footer is corrupted. I am reading multiple files from one directory using sparksql.</P><P>In that dir one file's footer is corrupted and so spark crashes. Is there any way to just ignore that corrupted blocks and read other</P><P> files as it is? I switched off the filter-pushdown by using sqlContext.setConf("spark.sql.parquet.filterPushdown","false")</P><P>Code used to read multiple files. (Here, /data/tempparquetdata/br.1455148800.0 is corrupted )</P><P>val newDataDF = sqlContext.read.parquet("/data/tempparquetdata/data1.parquet","/data/tempparquetdata/data2.parquet","/data/tempparquetdata/br.1455148800.0")</P><P>newDataDF.show throws the Exception " java.lang.RuntimeException: hdfs://CRUX2-SETUP:9000/data/tempparquetdata/br.1455148800.0 is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [82, 52, 24, 10]"</P> Fri, 30 Dec 2016 10:01:46 GMT khyati_shah 2016-12-30T10:01:46Z