Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark failed to get database default

SOLVED Go to solution
Highlighted

Spark failed to get database default

Master Collaborator

Hi, 

 I just did a fresh clean install of CDH 5.11 and with Hive and Spark, everythin in dev mode - so embedded database, no HA, simple setup.

 

When I try to run spark-shell a got an exception:

 

Spark context available as sc (master = yarn-client, app id = application_1498152485620_0001).
17/06/22 19:30:59 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.1.0
17/06/22 19:30:59 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
SQL context available as sqlContext.

 

 

Is this some bug related to Spark 1.6?

 

Thanks

Tomas

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Spark failed to get database default

Champion

@mbigelowi agree with you on this.

 

Could you check your Metastore / HS2  status

 

sudo service hive-metastore status
sudo service hive-server2 status

 

6 REPLIES 6

Re: Spark failed to get database default

Master Collaborator
The interesting thing is, that if I download spark 2.1 and configure it (point to hadoop conf dir in etc), it just works ok in YARN a does not have a problem to show databases or show tables.

Re: Spark failed to get database default

Champion

The database it is trying to access is the backend to the Hive Metastore.  Are you able to access and view databases and tables in Hive?

Re: Spark failed to get database default

Master Collaborator

I am sorry, but probably my mistake. I dont know why but now the sqlContext.sql("show tables").collect gathers the tables so I am able to access the metastore. The warning message is still there during spark-shell startup. But it works. 

 

 

Re: Spark failed to get database default

Champion

@mbigelowi agree with you on this.

 

Could you check your Metastore / HS2  status

 

sudo service hive-metastore status
sudo service hive-server2 status

 

Re: Spark failed to get database default

New Contributor

Still not clear on what the solution is for this error.  We are getting the same error.  Please help !!

Re: Spark failed to get database default

Champion

Could you let me know what is the issue you are facing ? whats the error ?