Member since
03-14-2016
4721
Posts
1111
Kudos Received
874
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2823 | 04-27-2020 03:48 AM | |
| 5475 | 04-26-2020 06:18 PM | |
| 4651 | 04-26-2020 06:05 PM | |
| 3700 | 04-13-2020 08:53 PM | |
| 5602 | 03-31-2020 02:10 AM |
01-30-2020
06:24 AM
@mike_bronson7 Usually Postgres writesinflight data and peristent data files inside the following dir " /var/lib/pgsql/data/pg_xlog" # ls -lart /var/lib/pgsql/data/pg_xlog Do you see this file "000000010000003A00000023" there int he above output ...any permission issue .. Or has anyone deleted that file by mistake ? So the previously mentioned error indicates that Postgres is not starting successfully because of that missing file. FATAL: the database system is starting up
LOG: could not open file "pg_xlog/000000010000003A00000023" (log file 58, segment 35): No such file or directory
... View more
01-30-2020
06:11 AM
1 Kudo
@mike_bronson7 Ambari is not listening on port 8080 because before starting to listen on that port it has to be able to connect to the DB successfully. But in your case Postgres itself is not starting successfully that why we see these errors FATAL: the database system is starting up
LOG: could not open file "pg_xlog/000000010000003A00000023" (log file 58, segment 35): No such file or directory Please consult with your Postgres DB admin to get the Postgres fixed first it should ot be showing those FATAL errors (if those are the latest errors after postgres restart) then that needs to be fixed first.
... View more
01-30-2020
06:02 AM
@mike_bronson7 Usually we find Postgres Db logs inside " /var/lib/pgsql/data/pg_log/" dir
... View more
01-30-2020
05:59 AM
@mike_bronson7 As you mentioned test Db command " its hang ( not return prompt ) " This indicates some issue from Postgres side. Hence can you please try restarting Postgres DB once . then re run the Test command again. Still if you face any issue then please share the full Postgres log.
... View more
01-30-2020
05:56 AM
@mike_bronson7 As you are using embedded Local postgres instance hence you can replace "localhost" with `hostname -f` like following. Or just use the output of "hostname -f" command in the previously mentioned command to replace "localhost" # /usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://`hostname -f`:5432/ambari ambari bigdata org.postgresql.Driver .
... View more
01-30-2020
05:54 AM
@mike_bronson7 As mentioned earlier the PATH "/usr/lib/ambari-server/postgresql-42.2.2.jar" might be different in your casse. So please check what is the JAR file name in your case. # ls -lart /usr/lib/ambari-server/postgresql-*
... View more
01-30-2020
05:49 AM
1 Kudo
@mike_bronson7 Can you please try the following command to verify if Postgres is listening to * address and port is opened? # ps -ef | grep pgsql | grep ^postgres
# netstat -tnlpa | grep 5432
# netstat -tnlpa | grep $POSTGRES_PID If above does not show the port is listening properly then we might need to check the postgres logs. Next can you please check if the Following Test Postgres DB connection command is running without any issue on Ambari Server host? # /usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://localhost:5432/ambari ambari bigdata org.postgresql.Driver The please try to run the same command using HOSTNAME of PostgresDB (instead of "localhost") # /usr/jdk64/jdk1.8.0_112/bin/java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/lib/ambari-server/postgresql-42.2.2.jar org.apache.ambari.server.DBConnectionVerification jdbc:postgresql://localhost:5432/ambari ambari bigdata org.postgresql.Driver Please NOTE that the "/usr/lib/ambari-server/postgresql-42.2.2.jar" PATH and the JDK installation path might be different in your case so please update the command accordingly. . . Also as we see the message : REASON: Server not yet listening on http port 8080 after 50 seconds. Exiting. So it will be also good to try setting the following and then try to restart ambari again properto to a little high value like 120 or 150 seconds as sometimes ambari takes little longer time to start. Default value is 50 seconds. server.startup.web.timeout=150 .
... View more
01-30-2020
05:17 AM
1 Kudo
@dewi As we repeatedly see this WARNING: 2020-01-30T09:25:38,383 WARN [main]: metastore.RetryingMetaStoreClient (:()) - MetaStoreClient lost connection. Attempting to reconnect (10 of 24) after 5s. getCurrentNotificationEventId
org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId Hence can you please try this to see if this works for you? Login to Ambari UI --> Hive --> Configs (Tab) --> Custom hive-site.xml Click on "Add" property button and then add the following property: hive.metastore.event.db.notification.api.auth=false Then restart HiveServer2 . . Also in your log we see the following ERROR: 2020-01-30T09:25:38,225 ERROR [json-metric-reporter]: metrics2.JsonFileMetricsReporter (:()) - Unable to rename temp file /tmp/hmetrics242742500630655243json to /tmp/report.json So can you lease check what ios the permission and ownership on the mentioned file? It should be owned and writable by "hive:hadoop" user. Example: # ls -lart /tmp/report.json
-rw-r--r--. 1 hive hadoop 3300 Jan 30 13:16 /tmp/report.json .
... View more
01-30-2020
05:02 AM
@AarifAkhter While setting up your maria DB have you performed the step mentioned in the doc as https://docs.cloudera.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-administration/content/using_ambari_with_mysql.html You must pre-load the Ambari database schema into your MySQL database using the schema script. Run the script in the same location where you find the Ambari-DDL-MySQL-CREATE.sql file. You should find the Ambari-DDL-MySQL-CREATE.sql file in the /var/lib/ambari-server/resources/ directory of the Ambari Server host, after you have installed Ambari Server. Ambari also shows this kind of message whenuser performs "ambari-server setup" while setting up the database. WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql . Also please let us know if your MariaDB instance running on "localhost" Or on "ip-172-31-9-188.xxxxxxxxxxxxxxx.internal" Can you please change the "localhost" to the hostname of the DB. server.jdbc.hostname=ip-172-31-9-188.xxxxxxxxxxxxxxx.internal Also please verify if the "MariaDB" exist on the mariaDB running on host 'ip-172-31-9-188.xxxxxxxxxxxxxxx.internal' # mysql -u ambari -p
Enter password:
show databases; .
... View more
01-30-2020
04:32 AM
@AarifAkhter For MariaDB as well can you chose the --jdbc-db=mysql while setting up the Ambari DB. Or please update the ambari.properties to mysql where you might have selected as MariaDB server.jdbc.database=mysql https://github.com/apache/ambari/blob/branch-2.7/ambari-server/src/main/python/ambari_server/dbConfiguration.py#L49 Reference Doc: https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/administering-ambari/content/amb_using_ambari_with_mysql_or_mariadb.html
... View more