Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark2 Can't write dataframe to parquet hive table : HiveFileFormat`. It doesn't match the specified format `ParquetFileFormat`.

avatar
New Contributor

I'm trying to save dateframe in table hive,:

.toDF()
.repartition($"col", $"col2", $"col3", $"col4")
.write
.format("parquet")
.mode(saveMode)
.partitionBy("col", "col2", "col3", "col4")
.saveAsTable("db.bloc_views")

But i get this error.

The format of the existing table project_bsc_dhr.bloc_views is `HiveFileFormat`. It doesn't match the specified format `ParquetFileFormat`.; org.apache.spark.sql.AnalysisException: The format of the existing table project_bsc_dhr.bloc_views is `HiveFileFormat`. It doesn't match the specified format `ParquetFileFormat`.;

1 REPLY 1

avatar
New Contributor

Try change this:

.format("parquet")

to this:

.format("hive")