question Spark failed to get database default in Archives of Support Questions (Read Only)

Spark failed to get database default

Tomas79 — Fri, 16 Sep 2022 11:48:34 GMT

Hi,

I just did a fresh clean install of CDH 5.11 and with Hive and Spark, everythin in dev mode - so embedded database, no HA, simple setup.

When I try to run spark-shell a got an exception:

Spark context available as sc (master = yarn-client, app id = application_1498152485620_0001).
17/06/22 19:30:59 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.1.0
17/06/22 19:30:59 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
SQL context available as sqlContext.

Is this some bug related to Spark 1.6?

Thanks

Tomas

Re: Spark failed to get database default

Tomas79 — Thu, 22 Jun 2017 18:01:41 GMT

The interesting thing is, that if I download spark 2.1 and configure it (point to hadoop conf dir in etc), it just works ok in YARN a does not have a problem to show databases or show tables.

Re: Spark failed to get database default

mbigelow — Fri, 23 Jun 2017 01:46:31 GMT

The database it is trying to access is the backend to the Hive Metastore. Are you able to access and view databases and tables in Hive?

Re: Spark failed to get database default

csguna — Fri, 23 Jun 2017 05:03:59 GMT

@mbigelowi agree with you on this.

Could you check your Metastore / HS2 status

sudo service hive-metastore status
sudo service hive-server2 status

Re: Spark failed to get database default

Tomas79 — Fri, 23 Jun 2017 09:59:17 GMT

I am sorry, but probably my mistake. I dont know why but now the sqlContext.sql("show tables").collect gathers the tables so I am able to access the metastore. The warning message is still there during spark-shell startup. But it works.

Re: Spark failed to get database default

MuniraG — Mon, 03 Jun 2019 14:48:52 GMT

Still not clear on what the solution is for this error. We are getting the same error. Please help !!

Re: Spark failed to get database default

csguna — Wed, 05 Jun 2019 07:15:07 GMT

Could you let me know what is the issue you are facing ? whats the error ?