Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

Solved Go to solution

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

Mentor

@Adil BAKKOURI

The below error tells me 2 things you are either using a MySQL server which is not supported or your jdbc driver version is not correct can you validate your Mysql database against the HXW support matrix


2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful.

Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'.

The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.

ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

Command failed after 1 tries


108031-db-versions.png

I filtered the above for HDP 2.6

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

Mentor

@Adil BAKKOURI

Can you firstly stop/ re-run ambari with the below command


ambari-server stop

Then

ambari-server start --skip-database-check

Your Ambari database has many orphaned config please see this Jira how to clean you Ambari database though it would be faster to start with a clean installation but it good to get your hands dirty !!!

https://issues.apache.org/jira/browse/AMBARI-20875


CREATE TEMPORARY TABLE IF NOT EXISTS orphaned_configs AS (SELECT cc.config_id, cc.type_name, cc.version_tag FROM clusterconfig cc, clusterconfigmapping ccm WHERE cc.config_id NOT IN (SELECT scm.config_id FROM serviceconfigmapping scm) AND cc.type_name != 'cluster-env' AND cc.type_name = ccm.type_name AND cc.version_tag = ccm.version_tag);

Followed by

DELETE FROM clusterconfigmapping WHERE EXISTS (SELECT 1 FROM orphaned_configs WHERE clusterconfigmapping.type_name = orphaned_configs.type_name AND clusterconfigmapping.version_tag = orphaned_configs.version_tag);

Followed by

DELETE FROM clusterconfig WHERE clusterconfig.config_id IN (SELECT config_id FROM orphaned_configs);

Followed by

SELECT * FROM orphaned_configs;

Then

DROP TABLE orphaned_configs;


Now you Ambari db should be clean to proceed with the hive setup

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

New Contributor

hello,
Mysql-connector-java is updated

107981-connector-update.jpg



ambari-server driver and db :

107973-connector-ambari-server.jpg


still have the warning on my ambari-server start

107982-connector-db-check.jpg



and finally the failed connection rom ambari :

107974-still-not.jpg



and here's the stderr & stdout :

stderr:

  1. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  2. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  3. Traceback (most recent call last): File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 530, in <module> CheckHost().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute method(env) File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 207, in actionexecute raise Fail(error_message) resource_management.core.exceptions.Fail: Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  4. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

stdout:

  1. 2019-04-16 15:55:43,864 - Host checks started. 2019-04-16 15:55:43,864 - Check execute list: db_connection_check 2019-04-16 15:55:43,864 - DB connection check started. WARNING: File /var/lib/ambari-agent/cache/DBConnectionVerification.jar already exists, assuming it was downloaded before WARNING: File /var/lib/ambari-agent/cache/mysql-connector-java.jar already exists, assuming it was downloaded before 2019-04-16 15:55:43,866 - call['/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-agent/cache/DBConnectionVerification.jar:/var/lib/ambari-agent/cache/mysql-connector-java.jar -Djava.library.path=/var/lib/ambari-agent/cache org.apache.ambari.server.DBConnectionVerification "jdbc:mysql://master.rh.bigdata.cluster/hive" "hive" [PROTECTED] com.mysql.jdbc.Driver'] {} 2019-04-16 15:55:44,473 - call returned (1, "Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.\nERROR: Unable to connect to the DB. Please check DB connection properties.\ncom.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.") 2019-04-16 15:55:44,474 - DB connection check completed. 2019-04-16 15:55:44,476 - Host checks completed. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  2. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  3. Command failed after 1 tries


Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

New Contributor

hello,
Mysql-connector-java is updated

107981-connector-update.jpg



ambari-server driver and db :

107973-connector-ambari-server.jpg


still have the warning on my ambari-server start

107982-connector-db-check.jpg



and finally the failed connection rom ambari :

107974-still-not.jpg



and here's the stderr & stdout :

stderr:
2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure 
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. 
Traceback (most recent call last):  File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 530, in <module>    CheckHost().execute()  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute    method(env)  File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 207, in actionexecute    raise Fail(error_message) resource_management.core.exceptions.Fail: Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure 
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
stdout:
2019-04-16 15:55:43,864 - Host checks started. 2019-04-16 15:55:43,864 - Check execute list: db_connection_check 2019-04-16 15:55:43,864 - DB connection check started. WARNING: File /var/lib/ambari-agent/cache/DBConnectionVerification.jar already exists, assuming it was downloaded before WARNING: File /var/lib/ambari-agent/cache/mysql-connector-java.jar already exists, assuming it was downloaded before 2019-04-16 15:55:43,866 - call['/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-agent/cache/DBConnectionVerification.jar:/var/lib/ambari-agent/cache/mysql-connector-java.jar -Djava.library.path=/var/lib/ambari-agent/cache org.apache.ambari.server.DBConnectionVerification "jdbc:mysql://master.rh.bigdata.cluster/hive" "hive" [PROTECTED] com.mysql.jdbc.Driver'] {} 2019-04-16 15:55:44,473 - call returned (1, "Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.\nERROR: Unable to connect to the DB. Please check DB connection properties.\ncom.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.") 2019-04-16 15:55:44,474 - DB connection check completed. 2019-04-16 15:55:44,476 - Host checks completed. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure 
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. 
Command failed after 1 tries

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

New Contributor

hello,
Mysql-connector-java is updated

107981-connector-update.jpg



ambari-server driver and db :

107973-connector-ambari-server.jpg


still have the warning on my ambari-server start

107982-connector-db-check.jpg



and finally the failed connection rom ambari :

107974-still-not.jpg



and here's the stderr & stdout :

stderr:

  1. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  2. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  3. Traceback (most recent call last): File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 530, in <module> CheckHost().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute method(env) File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 207, in actionexecute raise Fail(error_message) resource_management.core.exceptions.Fail: Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  4. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

stdout:

  1. 2019-04-16 15:55:43,864 - Host checks started. 2019-04-16 15:55:43,864 - Check execute list: db_connection_check 2019-04-16 15:55:43,864 - DB connection check started. WARNING: File /var/lib/ambari-agent/cache/DBConnectionVerification.jar already exists, assuming it was downloaded before WARNING: File /var/lib/ambari-agent/cache/mysql-connector-java.jar already exists, assuming it was downloaded before 2019-04-16 15:55:43,866 - call['/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-agent/cache/DBConnectionVerification.jar:/var/lib/ambari-agent/cache/mysql-connector-java.jar -Djava.library.path=/var/lib/ambari-agent/cache org.apache.ambari.server.DBConnectionVerification "jdbc:mysql://master.rh.bigdata.cluster/hive" "hive" [PROTECTED] com.mysql.jdbc.Driver'] {} 2019-04-16 15:55:44,473 - call returned (1, "Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.\nERROR: Unable to connect to the DB. Please check DB connection properties.\ncom.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.") 2019-04-16 15:55:44,474 - DB connection check completed. 2019-04-16 15:55:44,476 - Host checks completed. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  2. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  3. Command failed after 1 tries


Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

Mentor

@Adil BAKKOURI

The attached /var/log/ambari-server/ambari-server-check-database.log shows you are trying to connect to postgres which is the embedded database.

My guess is your ambari Server is using the embedded postgres database and NOT MySQL so what you need to do after installing and configuring MySQL and Hive is

Install the mySQL driver adapt the commands according to your OS

yum install -y mysql-connector-java

Make the jars available for hive etc this won't effect the Postgres installation

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

Now continue with the Ambari hive setup it should succeed NOW!!!!

Highlighted

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

New Contributor

Hello Geoffrey Shelton Okot ,


I tried with the "an existing Mysql database" but still have the same error, i also tried with the ip address and the localhost in the URL of the database but still nothing! here a screenshot of my conf

107956-existing-db.jpg


PS : i cant see your screenshot can you upload it again ?


Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

New Contributor

Hi Geoffrey Shelton Okot ,

first I want to thank you for your answer,

I deleted hive and restarted all instalations and configurations as you said and the tuto explain but still the same error at the end.

Here some screenshoots of the conf:

mysql and systeme time : 107943-time-mysql.jpg

user and database on mysql :


107922-usr-database-mysql.jpg

hive conf on ambari :

107953-hive-conf.jpg




I found this warning on /var/log/ambari-server/ambari-server-check-database.log :

107938-checkdb-mysql.jpg

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

Mentor

@Adil BAKKOURI

You have done everything correctly but you are not selecting the existing database you created See attached screenshot all should work !!!

107954-mysql-hive-setup.png

Please use the values of the hive DB created previously it should pick the correct URL and host !!

Re: HiveMetastore & Hiveserver2 Failed to start On HortonWorks Hadoop cluster ?

New Contributor

hello,
Mysql-connector-java is updated

107981-connector-update.jpg



ambari-server driver and db :

107973-connector-ambari-server.jpg


still have the warning on my ambari-server start

107982-connector-db-check.jpg



and finally the failed connection rom ambari :

107974-still-not.jpg



and here's the stderr & stdout :

stderr:

  1. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  2. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  3. Traceback (most recent call last): File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 530, in <module> CheckHost().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute method(env) File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 207, in actionexecute raise Fail(error_message) resource_management.core.exceptions.Fail: Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  4. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

stdout:

  1. 2019-04-16 15:55:43,864 - Host checks started. 2019-04-16 15:55:43,864 - Check execute list: db_connection_check 2019-04-16 15:55:43,864 - DB connection check started. WARNING: File /var/lib/ambari-agent/cache/DBConnectionVerification.jar already exists, assuming it was downloaded before WARNING: File /var/lib/ambari-agent/cache/mysql-connector-java.jar already exists, assuming it was downloaded before 2019-04-16 15:55:43,866 - call['/usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-agent/cache/DBConnectionVerification.jar:/var/lib/ambari-agent/cache/mysql-connector-java.jar -Djava.library.path=/var/lib/ambari-agent/cache org.apache.ambari.server.DBConnectionVerification "jdbc:mysql://master.rh.bigdata.cluster/hive" "hive" [PROTECTED] com.mysql.jdbc.Driver'] {} 2019-04-16 15:55:44,473 - call returned (1, "Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.\nERROR: Unable to connect to the DB. Please check DB connection properties.\ncom.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.") 2019-04-16 15:55:44,474 - DB connection check completed. 2019-04-16 15:55:44,476 - Host checks completed. 2019-04-16 15:55:44,477 - Check db_connection_check was unsuccessful. Exit code: 1. Message: Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. ERROR: Unable to connect to the DB. Please check DB connection properties. com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
  2. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  3. Command failed after 1 tries


Don't have an account?
Coming from Hortonworks? Activate your account here