Databricks Error Inquiry: org.apache.spark.SparkException: Unable to fetch tables of db default

@JN_000 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Spark expert @Bharati  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Thank you so much!

We verified the same in the CDP environment, as we are uncertain about the Databricks Spark environment. 

As we have mixed of managed and external tables , extracted the necessary information through HWC. 

>>> database=spark.sql("show tables in default").collect()
23/07/20 10:04:45 INFO rule.HWCSwitchRule: Registering Listeners
23/07/20 10:04:47 WARN conf.HiveConf: HiveConf of name hive.masking.algo does not exist
Hive Session ID = e6f70006-0c2e-4237-9a9e-e1d19901af54
>>> desiredColumn="name"
>>> tablenames = []
>>> for row in database:
...  cols = spark.table(row.tableName).columns
...  listColumns= spark.table(row.tableName).columns
...  if desiredColumn in listColumns:
...   tablenames.append(row.tableName)
...
>>>
>>> print("\n".join(tablenames))
movies
tv_series_abc
cdp1
tv_series
spark_array_string_example
>>>