pyspark - can not create managed table

[#1]

I'm writing some pyspark code where I have a dataframe that I want to write to a hive table.

I'm using a command like this.

dataframe.write.mode("overwrite").saveAsTable(“bh_test”)

Everything I've read online indicates that this should, by default, create  a managed table.

However, no matter what I try, it always creates an external table. Is there a configuration setting somewhere that overrides the default behavior?

[#2 · Accepted solution]

Hi @haze5736, You need to use Hive Warehouse Connector (HWC) software to query Apache Hive managed tables from Apache Spark. Using HWC API you can read and write Apache Hive tables from Apache Spark. For example, to write the managed table. 

df.write.format(HiveWarehouseSession().HIVE_WAREHOUSE_CONNECTOR).option("table", &tableName>).option("partition", <partition_spec>).save()

Ref: https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/integrating-hive-and-bi/topics/hive-read-writ...

For more details you can refer the below documentation: 
https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/integrating-hive-and-bi/topics/hive_hivewareh...
https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/integrating-hive-and-bi/topics/hive_submit_a_...

[#3]

@haze5736 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks