Member since
06-24-2019
6
Posts
0
Kudos Received
0
Solutions
07-14-2019
04:19 AM
Hi @bgooley Thanks for the reply. I checked the var/log/cloudera-scm-server.log. Sharing the exception part at the time when vm starts at below link:- cloudera-scm-server.log Sorry for the big log but posted all so that no important detail is missed. Further, I again confirmed that cloudera-scm-server is not up. It fails to keep up since the start of the issue. I checked the process cmf.Main, it doesnt exist! Since I am facing this issue, cannot work on VM really. So since VM is the freshly started with this issue, so no scope for any long-running/hung queries cant be there! This is verified here:- mysql> SHOW FULL PROCESSLIST; +----+------+-----------+------+---------+------+-------+-----------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+-----------+------+---------+------+-------+-----------------------+ | 2 | root | localhost | NULL | Query | 0 | NULL | SHOW FULL PROCESSLIST | +----+------+-----------+------+---------+------+-------+-----------------------+ 1 row in set (0.00 sec) Please suggest, the steps for resolving the issue ..(any hint from cloudera-scm-server.log) I am stuck with this issue since 10 days.:( Thanks in advance.
... View more
07-14-2019
03:16 AM
Hey @lwang, Thanks for reply. I tried to run the command as below:- [root@quickstart ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm
Enter SCM password:
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/java/jdk1.7.0_67-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
log4j:ERROR Could not find value for key log4j.appender.A
log4j:ERROR Could not instantiate appender named "A".
[2019-07-14 15:30:28,099] INFO 0[main] - com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java) - Successfully connected to database.
All done, your SCM database is configured correctly!
[root@quickstart ~]#
[root@quickstart ~]# service cloudera-scm-server status
cloudera-scm-server dead but pid file exists But my problem is still there after this! I have no way to understand what causes this issue 😞 Please help!
... View more
07-04-2019
06:57 AM
Hi @lwang, I am petrified and fearful now. When I look into location:- /opt/cloudera/ there is no directory as "cm" there! What i find there are below directiories:- [root@quickstart cloudera]# pwd /opt/cloudera [root@quickstart cloudera]# ls csd parcel-cache parcel-repo parcels [root@quickstart cloudera]# What could be the real issue behind this? Please help!
... View more
07-02-2019
11:51 PM
HI @lwang , Thanks for your reply. Yes, the timestamp is updated for log. I actually re-checked the db.properties. It contains below properties:- com.cloudera.cmf.db.type=mysql com.cloudera.cmf.db.host=localhost com.cloudera.cmf.db.name=cm com.cloudera.cmf.db.user=cm com.cloudera.cmf.db.setupType=EXTERNAL com.cloudera.cmf.db.password=cloudera With above credentials, mysql is able connect. NO issues there. My default db is mysql and I have never installed postgresql nor upgraded the CDH setup yet. I have configured for single node cluster. Till the last time when cloudera manager had worked, the used services were : HDFS,Hive, Zookeeper, Spark, Impala. What can be the issue then? Please help!
... View more
07-02-2019
01:04 AM
Clouder team, your help is needed badly on this issue; I am stuck because of issue. Please let me know if any details are needed. Thanks in advance.
... View more
06-28-2019
02:15 AM
Hi All,
Clouder Manager UI doesnt start at all. Tried opening localhost:7180 or 127.0.0.1:7180, nothing worked.
Opening a new issue since the exception for cloudera-scm-server is different in my case and none of solutions in previous thread really matched.
Cloudera manager worked till yesterday and I applied no update till today when its UI is not startng at all.
I dont understand why issue started today suddenly. 😞
I checked other threads in this community and did following steps in set of attempts, but clouder-scm-server still fails to keep started:-
1. Checked Cloudera-scm-server status:
[root@quickstart cloudera-scm-server]# service cloudera-scm-server status cloudera-scm-server dead but pid file exists
2. Removed pid as
rm /var/run/cloudera-scm-server.pid 3. Tried to stop cloudera-scm-server-db, but got issue!
[cloudera@quickstart ~]$ sudo service cloudera-scm-server-db stop cloudera-scm-server-db: unrecognized service
4. One of the thread suggested that postgresql might not be running,so i checked in /etc/rc.d/init.d/ for postgresql.
But its not there!
[root@quickstart ~]# service postgresql start postgresql: unrecognized service
5. I checked in /etc/hosts if loopback address is missing 127.0.0.1 localhost. But the entry is already there.
6. Checked if port 7180 is listening by command:
netstat -na | grep 7180
But I get no output here.
7. I checked /var/log/cloudera-scm-server/cloudera-scm-server.out for an issue, there I get below exception:-
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Exception in thread "MainThread" javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Could not open connection
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310)
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1397)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:62)
at com.cloudera.enterprise.AbstractWrappedEntityManager.beginForRollbackAndReadonly(AbstractWrappedEntityManager.java:89)
at com.cloudera.cmf.persist.CmfEntityManager.beginForRollbackAndReadonly(CmfEntityManager.java:347)
at com.cloudera.server.cmf.Main.initializeCustomQueryCache(Main.java:705)
at com.cloudera.server.cmf.Main.<init>(Main.java:307)
at com.cloudera.server.cmf.Main.main(Main.java:216)
Caused by: org.hibernate.exception.JDBCConnectionException: Could not open connection
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:132)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:160)
at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1426)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:59)
... 5 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 108,228 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3427)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3327)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
at com.mysql.jdbc.ConnectionImpl.setTransactionIsolation(ConnectionImpl.java:5133)
at com.mchange.v2.c3p0.impl.NewProxyConnection.setTransactionIsolation(NewProxyConnection.java:701)
at org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:86)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214)
... 10 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2914)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3337)
... 20 more
I do not understand precise meaning of this exception and root cause for the issue.
NOTE: Internet is working on VM
Please, please help suggest solution.
... View more
Labels:
- Labels:
-
Cloudera Manager
-
Quickstart VM