Support Questions
Find answers, ask questions, and share your expertise

SaveAsTable not creating table in hive metastore

New Contributor

I am trying to save json data in hive table as follows:

input.write.mode(SaveMode.Overwrite).format("json").partitionBy("col1", "col2").saveAsTable("data")

it is creating data under spark warehouse dir in HDFS but no table gets created in hive metastore. I am using spark 2.3.0. Can someone help with this?


Super Guru

@Aman Rastogi

Could you once make sure you have enabled hive support while creating spark session.

Ex: enabling hive support

val spark = SparkSession
  .config("spark.sql.warehouse.dir", warehouseLocation)

Refer to this link for more details regards enableHiveSupport.

New Contributor

@Shu, Thanks for responding. I tried to update ticket earlier but, couldn't happen. I am not sure if some configs which I had set were causing this issue but, after commenting those out it started working fine. Below are the configs:

.config("spark.memory.fraction", "0.3")
.config("", "0.8")
.config("spark.executor.heartbeatInterval", "300")
.config("", "900")
.config("spark.sql.autoBroadcastJoinThreshold", "-1")