Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP 2.3.4 Windows Installation and Derby Failure

Solved Go to solution
Highlighted

HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

After installing HDP 2.3.4 on windows server 2012, the following service does not start; DerbyServer. The service terminates almost immediately when trying to start. I see the below error message when looking at the Hive logs.

Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("org.apache.derby.jdbc.ClientDriver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.

I see the "derby-10.10.2.0.jar" file within the hdp\hive-xxx\lib" directory

Thanks for the help...

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

All,

I'm embarrassed to report the following... After nearly pulling the plug on this POC using windows server, i decided to start from the very beginning, basically "is it plugged in". While doing so, i discovered that I had fat fingered ip addresses in the Host File, which led to issues with the installation. After correcting the host IP addresses, i was able to successfully reinstall HDP common, Hive, Oozie, HBase.

Configuration:

2 windows servers (01 and 02)

Options: HDP Common, HBase, Hive and Oozie with MSSQL Metastore ~could never get DerbyServer to work

Modified Hive-Site.xml

Set dataNucleus.autoCreateSchema and dataNucleus.validateTables to True

Note: While monitoring the Hive and Hiveserver2 test, i noticed the test was blocked by an open connection from hive. It was an idle connection that prevented the hive test from creating the missing tables, like TBL_Privs.

10 REPLIES 10

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

Hi John,

I'm also going through the install of HDP 2.3.4 on windows, and have found that derby server does not start.

I downloaded the jar dependencies for derbynet-10.10.2.0.jar, derbyclient-10.10.2.0.jar and derbytools-10.10.2.0.jar and added them into the hive-version/lib directory.

In the hive/bin/derbyserver.xml I modified the class path to include the derbynet-10.10.2.0.jar and the derby server starts up. On restarting the hiveserver2 service, it connects to the derby server, however the DB tables have not been created so the next step is to check why hive isn't creating the tables, this maybe due to configuration, or if there is a script available to manually create the tables.

Most of the install for windows has worked, but I've found datanodes, and yarn nodemanagers did not startup automatically, and after some tweaking have got them working, did they startup automatically for you what did you're cluster properties look like?

Thanks,

Chris.

Edit: after a bit of trial and error, I managed to get the "Run-SmokeTest HiveServer2" to pass however, have not yet managed to get the "Run-SmokeTest Hive" to pass. I'm not certain whether the hive client will operate with the hiveserver2, as opposed to the beeline client. If anyone can advise whether the standard hive command should work against hiveserver2 instead of the beeline client that'd be good to know. Otherwise, it seems the beeline client passes the smoke test. Other useful resources for info on installation included the hive website https://cwiki.apache.org/confluence/display/Hive/HiveDerbyServerMode.

Re: HDP 2.3.4 Windows Installation and Derby Failure

Mentor

@Chris Davey excellent sleuthing, in the hive scripts folders, there are DDL scripts to create all usual tables for any of the DBs. See if derby is there. I'm going to wikify your answer.

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

Hi Artem,

After setting the datanucleas.autoCreateSchema to true in hive-site.xml it has bootstrapped the schema and hive server starts up (I can set that back to false now and restart the service).

Re: HDP 2.3.4 Windows Installation and Derby Failure

Mentor

You should document your steps for the rest of the community. @Chris Davey

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

I'll see if I can get through the entire install first. It doesn't look like the procedure I've used has allowed the smoke test for hive or hiveserver2 to pass yet. So back to the drawing board.

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

Hi Chris,

I found even after setting autoCreateSchema to true, i was still missing tables, specifically xxx_Privs. I ended up switching my install to use MSSQL with a clean database. So far, the tests were able to get past the initial failures, but they are still running. I'll post an update after the test completes.

BTW - does anyone know how to get Hive Web UI working? The logs reference a missing *.war file, which i can't find anywhere in the installation directories.

-John

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

After a lot of trial and error, i was able to get all the base HDP components working. However, I was never able to get the installer to successfully install GZIP, LZO, or Oozie Web Console.

I know HDP on windows was never intended to run in production, but the amount of hacking necessary to get a base install working on windows is unfortunate.

This experience causes concerns that when I go to build the production cluster on Linux that i will have to deal with similar issues.

Re: HDP 2.3.4 Windows Installation and Derby Failure

Mentor
@John Gartman

your concerns are understandable but I can say that most HDP installations are on Linux and a lot of customers run it in production. Try it out on Linux and let us know.

Re: HDP 2.3.4 Windows Installation and Derby Failure

New Contributor

All,

I'm embarrassed to report the following... After nearly pulling the plug on this POC using windows server, i decided to start from the very beginning, basically "is it plugged in". While doing so, i discovered that I had fat fingered ip addresses in the Host File, which led to issues with the installation. After correcting the host IP addresses, i was able to successfully reinstall HDP common, Hive, Oozie, HBase.

Configuration:

2 windows servers (01 and 02)

Options: HDP Common, HBase, Hive and Oozie with MSSQL Metastore ~could never get DerbyServer to work

Modified Hive-Site.xml

Set dataNucleus.autoCreateSchema and dataNucleus.validateTables to True

Note: While monitoring the Hive and Hiveserver2 test, i noticed the test was blocked by an open connection from hive. It was an idle connection that prevented the hive test from creating the missing tables, like TBL_Privs.

Don't have an account?
Coming from Hortonworks? Activate your account here