Cloudera Community
amanr_rastogi
user rank
Explorer

Re: SaveAsTable not creating table in hive metasto...

by Explorer in Support Questions
‎05-15-2019 04:54 PM
‎05-15-2019 04:54 PM
@Shu, Thanks for responding. I tried to update ticket earlier but, couldn't happen. I am not sure if some configs which I had set were causing this issue but, after commenting those out it started working fine. Below are the configs: .config("spark.memory.fraction", "0.3") .config("spark.storage.memoryFraction", "0.8") .config("spark.executor.heartbeatInterval", "300") .config("spark.network.timeout", "900") .config("spark.sql.autoBroadcastJoinThreshold", "-1") ... View more

SaveAsTable not creating table in hive metastore

by Explorer in Support Questions
‎05-14-2019 02:50 AM
‎05-14-2019 02:50 AM
I am trying to save json data in hive table as follows: input.write.mode(SaveMode.Overwrite).format("json").partitionBy("col1", "col2").saveAsTable("data") it is creating data under spark warehouse dir in HDFS but no table gets created in hive metastore. I am using spark 2.3.0. Can someone help with this? ... View more
Contact Me
Online
Offline
Last Visited
‎07-03-2019 06:11 PM
Community Statistics
Member Since ‎05-13-2019 06:08 PM
Last Visited ‎07-03-2019 06:11 PM
Posts 4