Hi, I'm having an issue while upgrading the Cloudera External Postgres database(aws rds) from 9.5 to the latest 12.3. I did the below steps:
1. I'm using AWS RDS POSTGRES for Cloudera external DB. The PSQL 9.5 has end of life. So need to upgrade to the latest version of 12.3. We had to upgrade the copy 9.5 version to the latest 12.3. Both versions of rds psql roles, security, and databases are identical.
2. I've stopped the services that using the embedded database server and also ran the cloudera-scm-server stop command and I have changed the com.cloudera.cmf.db.host to a new version of rds psql.
3. After the above changes, I have started the cloudera-scm-server successfully with command but the UI didn't open, so I have checked the SCM logs then I found the below:
2021-02-13 05:12:07,164 WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1:com.mchange.v2.resourcepool.BasicResourcePool: com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@8326160 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (5). Last acquisition attempt exception:
org.postgresql.util.PSQLException: This ResultSet is closed.
at org.postgresql.jdbc2.AbstractJdbc2ResultSet.checkClosed(AbstractJdbc2ResultSet.java:2674)
at org.postgresql.jdbc2.AbstractJdbc2ResultSet.setFetchSize(AbstractJdbc2ResultSet.java:1791)
at org.postgresql.jdbc4.Jdbc4Statement.createResultSet(Jdbc4Statement.java:39)
at org.postgresql.jdbc2.AbstractJdbc2Statement$StatementResultHandler.handleResultRows(AbstractJdbc2Statement.java:211)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1772)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
at org.postgresql.jdbc2.AbstractJdbc2Connection.execSQLUpdate(AbstractJdbc2Connection.java:263)
at org.postgresql.jdbc2.AbstractJdbc2Connection.getTransactionIsolation(AbstractJdbc2Connection.java:775)
at com.mchange.v2.c3p0.impl.NewPooledConnection.<init>(NewPooledConnection.java:104)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:198)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014)
at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
2021-02-13 05:12:07,165 WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0:com.mchange.v2.resourcepool.BasicResourcePool: com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@8d16f8d -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (5). Last acquisition attempt exception:
org.postgresql.util.PSQLException: This ResultSet is closed.
4. I have never seen an error like that but as per my understanding, it is related to the Postgres JDBC jar file-related. I have searched the Postgres jar files related to the old version of 9.5 and found the below:
/usr/share/cmf/cloudera-navigator-server/jars/postgresql-9.0-801.jdbc4.jar
/usr/share/cmf/cloudera-navigator-server/libs/cdh5/postgresql-9.0-801.jdbc4.jar
/usr/share/cmf/common_jars/postgresql-9.0-801.jdbc4.jar
/usr/share/cmf/cloudera-scm-telepub/jars/postgresql-9.0-801.jdbc4.jar
/usr/share/cmf/cloudera-scm-telepub/libs/cdh5/postgresql-9.0-801.jdbc4.jar
/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/jars/postgresql-9.0-801.jdbc4.jar
/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/lib/impala/lib/postgresql-9.0-801.jdbc 4.jar
/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/lib/oozie/libserver/postgresql-9.0-801 .jdbc4.jar
/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/lib/oozie/libtools/postgresql-9.0-801.jdbc4.jar
Can someone help me to replace the old jar version with new version jar files. I know it is not simple to replace files just like that but do I really need to stop the cluster and do configs/ clean/ cache? I still want to keep the current version of CDH 5.13 for now. I have no idea what to do now. Can someone please help with the next steps?
I really appreciate your time and help.
Thanks,
Venkat
