Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDP 2.5 Sandbox - Ambari Server Wont Start

avatar
Super Collaborator

Hello!

Just downloaded the HDP2.5 sandbox and successfully started the vm.

ambari-server won't start after following the guides here.

Attaching the logs.

/var/log/ambari-server/ambari-server.log

30 Mar 2017 17:28:39,533 ERROR [main] AmbariServer:927 - Failed to run the Ambari Server
javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: could not read block 0 of relation base/16384/16567: read only 0 of 8192 bytes
Error Code: 0
Call: SELECT cluster_id, current_cluster_state, current_stack_id FROM clusterstate WHERE (cluster_id = ?)
	bind => [1 parameter bound]
Query: ReadObjectQuery(name="clusterStateEntity" referenceClass=ClusterStateEntity )
	at org.eclipse.persistence.internal.jpa.QueryImpl.getDetailedException(QueryImpl.java:382)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:260)
	at org.eclipse.persistence.internal.jpa.QueryImpl.getResultList(QueryImpl.java:473)
	at org.apache.ambari.server.orm.dao.ClusterDAO.findAll(ClusterDAO.java:92)
	at org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:53)
	at org.apache.ambari.server.state.cluster.ClustersImpl.loadClustersAndHosts(ClustersImpl.java:198)
	at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
	at org.apache.ambari.server.state.cluster.ClustersImpl.checkLoaded(ClustersImpl.java:187)
	at org.apache.ambari.server.state.cluster.ClustersImpl.getClusters(ClustersImpl.java:665)
	at org.apache.ambari.server.api.services.AmbariMetaInfo.reconcileAlertDefinitions(AmbariMetaInfo.java:1088)
	at org.apache.ambari.server.controller.AmbariServer.run(AmbariServer.java:593)
	at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:925)
Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: could not read block 0 of relation base/16384/16567: read only 0 of 8192 bytes
Error Code: 0
Call: SELECT cluster_id, current_cluster_state, current_stack_id FROM clusterstate WHERE (cluster_id = ?)
	bind => [1 parameter bound]
Query: ReadObjectQuery(name="clusterStateEntity" referenceClass=ClusterStateEntity )
	at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340)
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:684)
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:560)
	at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2055)
	at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:570)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:242)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)
	at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:714)
	at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2803)
	at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2756)
	at org.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:555)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1175)
	at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:904)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1134)
	at org.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:441)
	at org.eclipse.persistence.internal.sessions.AbstractSession.internalExecuteQuery(AbstractSession.java:3270)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1857)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1839)
	at org.eclipse.persistence.internal.indirection.NoIndirectionPolicy.valueFromQuery(NoIndirectionPolicy.java:326)
	at org.eclipse.persistence.mappings.ForeignReferenceMapping.valueFromRowInternal(ForeignReferenceMapping.java:2334)
	at org.eclipse.persistence.mappings.OneToOneMapping.valueFromRowInternal(OneToOneMapping.java:1848)
	at org.eclipse.persistence.mappings.ForeignReferenceMapping.valueFromRow(ForeignReferenceMapping.java:2178)
	at org.eclipse.persistence.mappings.ForeignReferenceMapping.readFromRowIntoObject(ForeignReferenceMapping.java:1505)
	at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildAttributesIntoObject(ObjectBuilder.java:462)
	at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:1005)
	at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:899)
	at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:852)
	at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:735)
	at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:689)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:805)
	at org.eclipse.persistence.queries.ReadAllQuery.registerResultInUnitOfWork(ReadAllQuery.java:962)
	at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:573)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1175)
	at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:904)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1134)
	at org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:460)
	at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1222)
	at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2896)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1857)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1839)
	at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)
	at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:258)
	... 10 more
Caused by: org.postgresql.util.PSQLException: ERROR: could not read block 0 of relation base/16384/16567: read only 0 of 8192 bytes
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:559)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:302)
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:1009)
	at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:644)
	... 50 more

/var/log/ambari-agent/ambari-agent.out

OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0

Any help is highly appreciated!

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Michael Dennis "MD" Uanang

The following error indicates that your few DB tables are corrupted.

org.postgresql.util.PSQLException: ERROR: could not read block 0 of relation base/16384/16567: read only 0 of 8192 bytes

.

The easiest way to verify the same is to just try doing a select query on the same table from the DB.

Exampel:

# psql -U ambari ambari
Password for user ambari:  bigdata

ambari=> SELECT * FROM clusterstate;

.

A similar discussion happened in the thread: https://community.hortonworks.com/questions/84010/ambari-server-fails-to-start-after-database-consis...

.

Or if you have earlier DB dump backup then you can restore the DB dump.

View solution in original post

7 REPLIES 7

avatar
Master Mentor

@Michael Dennis "MD" Uanang

The following error indicates that your few DB tables are corrupted.

org.postgresql.util.PSQLException: ERROR: could not read block 0 of relation base/16384/16567: read only 0 of 8192 bytes

.

The easiest way to verify the same is to just try doing a select query on the same table from the DB.

Exampel:

# psql -U ambari ambari
Password for user ambari:  bigdata

ambari=> SELECT * FROM clusterstate;

.

A similar discussion happened in the thread: https://community.hortonworks.com/questions/84010/ambari-server-fails-to-start-after-database-consis...

.

Or if you have earlier DB dump backup then you can restore the DB dump.

avatar
Super Collaborator

Thanks @Jay SenSharma.

This is a fresh download vbox sandbox.

Just did ambari-server reset 🙂

But now, I can't finish the setup having issues with HDP UTILS repo.

Failed to set locale, defaulting to C
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
Trying other mirror.

avatar
Master Mentor

@Michael Dennis "MD" Uanang

I see there is some issue. Let me check with the team to get it fixed ASAP.

avatar
Master Mentor

@Michael Dennis "MD" Uanang

Its working now. Please try again. Issue is fixed.

avatar
Super Collaborator

Since this is a fresh download, I just went to recreate the VM with a fresh copy of vdisk and it went OK. Thanks!

avatar
Explorer

Issue is resolved. We have fixed the broken link issue. Please verify.

avatar
Super Collaborator

Will post later, on the road. 🙂