https://community.cloudera.com/t5/Support-Questions/SPARK-Dataframe-and-IMPALA-CREATE-TABLE-issue/m-p/55991#M22628 if writing to parquet you just have to do something like: df.write.mode("append").parquet("/user/hive/warehouse/Mytable") and if you want to prevent the "small file" problem: df.coalesce(1).write.mode("append").parquet("/user/hive/warehouse/Mytable") Fri, 16 Jun 2017 07:24:49 GMT vanhalen 2017-06-16T07:24:49Z