Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark2 (STS) Derby

Highlighted

Spark2 (STS) Derby

New Contributor

Hello friends!

I am researching the Spark2 as a part of HDP 2.6.5. The most interesting part is STS, which provides SQL interface via JDBC. But I want to have a possibility to organize concurrent access to STS from different client, but STS works on Derby. Derby itself allows to have only one connection at the moment. So, is it possible to switch the installed STS to PostgreSQL instead of Derby? What is a recommended approach to organize a concurrent access to STS?

3 REPLIES 3
Highlighted

Re: Spark2 (STS) Derby

Expert Contributor

Please check the configuration. In HDP, STS doesn't use Derby by default. If the configuration is correct, STS should connect to Hive Metastore Service (which is running on MySQL/Postgres) instead.

Highlighted

Re: Spark2 (STS) Derby

New Contributor

Actually you are right, it uses my hive metastore running on top of PostgreSQL, and I can see all objects created in Hive before. But I saw in log-file mention about derby, if I rename derby jar-file, my sts installation can not start. And additionally to that, I am able to connect to the STS only via beeline installed locally on the same host with STS. All remote connections via DBeaver which is a JDBC-client are refused. At the same time I am able to connect to Hive with DBeaver. So, I strongly believe that internally it still uses derby.

Highlighted

Re: Spark2 (STS) Derby

New Contributor

This is the log entry that I am talking about:

INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 
INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 
INFO ObjectStore: Initialized ObjectStore
Don't have an account?
Coming from Hortonworks? Activate your account here