Member since
07-18-2018
53
Posts
0
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1790 | 07-30-2018 04:19 PM | |
| 4462 | 07-26-2018 06:53 PM |
08-21-2019
05:40 PM
Thanks for the quick response. I had firewall disabled and nc -v command seems working. [root@msl-dpe-perf80-100g resources]# java -cp /var/lib/ambari-server/resources/DBConnectionVerification.jar:/usr/share/java/mysql-connector-java.jar org.apache.ambari.server.DBConnectionVerification "jdbc:mysql://msl-dpe-perf80-100g.msl.lab:3306/ambari" "ambari" "bigdata" com.mysql.jdbc.Driver
ERROR: Unable to connect to the DB. Please check DB connection properties.
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
[root@msl-dpe-perf80-100g resources]# firewall-cmd --state
not running
[root@msl-dpe-perf80-100g resources]# telnet msl-dpe-perf80-100g.msl.lab 3306
Trying 10.100.10.80...
Connected to msl-dpe-perf80-100g.msl.lab.
Escape character is '^]'.
J
8.0.17\ufffdv2RS3\ufffdKwrx?^]7qj6caching_sha2_passwordConnection closed by foreign host.
[root@msl-dpe-perf80-100g resources]# nc -v msl-dpe-perf80-100g.msl.lab 3306
Ncat: Version 6.40 ( <a href="http://nmap.org/ncat" target="_blank">http://nmap.org/ncat</a> )
Ncat: Connected to 10.100.10.80:3306.
J
8.0.17\ufffdzZbd\ufffd\ufffd\ufffd\ufffd\ufffds_N}6+X?[|^caching_sha2_password
Ncat: Broken pipe. From command line, I can access the mysql. See below for detail [root@msl-dpe-perf80-100g resources]# mysql -uambari -pbigdata
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 172
Server version: 8.0.17 MySQL Community Server - GPL
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| ambari |
| druid |
| information_schema |
| mysql |
| performance_schema |
| registry |
| streamline |
| superset |
| sys |
+--------------------+
9 rows in set (0.01 sec)
mysql> use ambari;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+-------------------------------+
| Tables_in_ambari |
+-------------------------------+
| ClusterHostMapping |
| QRTZ_BLOB_TRIGGERS |
| QRTZ_CALENDARS |
| QRTZ_CRON_TRIGGERS |
| QRTZ_FIRED_TRIGGERS |
| QRTZ_JOB_DETAILS |
| QRTZ_LOCKS |
| QRTZ_PAUSED_TRIGGER_GRPS |
| QRTZ_SCHEDULER_STATE |
| QRTZ_SIMPLE_TRIGGERS |
| QRTZ_SIMPROP_TRIGGERS |
| QRTZ_TRIGGERS |
| adminpermission |
| adminprincipal |
| adminprincipaltype |
| adminprivilege |
| adminresource |
| adminresourcetype |
| alert_current |
| alert_definition |
| alert_group |
| alert_group_target |
| alert_grouping |
| alert_history |
| alert_notice |
| alert_target |
| alert_target_states |
| ambari_configuration |
| ambari_operation_history |
| ambari_sequences |
| artifact |
| blueprint |
| blueprint_configuration |
| blueprint_setting |
| clusterconfig |
| clusters |
| clusterservices |
| clusterstate |
| confgroupclusterconfigmapping |
| configgroup |
| configgrouphostmapping |
| execution_command |
| extension |
| extensionlink |
| host_role_command |
| host_version |
| hostcomponentdesiredstate |
| hostcomponentstate |
| hostconfigmapping |
| hostgroup |
| hostgroup_component |
| hostgroup_configuration |
| hosts |
| hoststate |
| kerberos_descriptor |
| kerberos_keytab |
| kerberos_keytab_principal |
| kerberos_principal |
| key_value_store |
| kkp_mapping_service |
| metainfo |
| permission_roleauthorization |
| remoteambaricluster |
| remoteambariclusterservice |
| repo_applicable_services |
| repo_definition |
| repo_os |
| repo_tags |
| repo_version |
| request |
| requestoperationlevel |
| requestresourcefilter |
| requestschedule |
| requestschedulebatchrequest |
| role_success_criteria |
| roleauthorization |
| servicecomponent_version |
| servicecomponentdesiredstate |
| serviceconfig |
| serviceconfighosts |
| serviceconfigmapping |
| servicedesiredstate |
| setting |
| stack |
| stage |
| topology_host_info |
| topology_host_request |
| topology_host_task |
| topology_hostgroup |
| topology_logical_request |
| topology_logical_task |
| topology_request |
| upgrade |
| upgrade_group |
| upgrade_history |
| upgrade_item |
| user_authentication |
| users |
| viewentity |
| viewinstance |
| viewinstancedata |
| viewinstanceproperty |
| viewmain |
| viewparameter |
| viewresource |
| viewurl |
| widget |
| widget_layout |
| widget_layout_user_widget |
+-------------------------------+
109 rows in set (0.00 sec)
mysql> SELECT User FROM mysql.user;
+------------------+
| User |
+------------------+
| ambari |
| druid |
| rangerdba |
| registry |
| streamline |
| superset |
| ambari |
| mysql.infoschema |
| mysql.session |
| mysql.sys |
| rangerdba |
| root |
| ambari |
| rangerdba |
+------------------+
14 rows in set (0.00 sec)
mysql> SHOW GRANTS FOR 'ambari'@'msl-dpe-perf80-100g.msl.lab';
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for ambari@msl-dpe-perf80-100g.msl.lab |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `ambari`@`msl-dpe-perf80-100g.msl.lab` |
| GRANT APPLICATION_PASSWORD_ADMIN,AUDIT_ADMIN,BACKUP_ADMIN,BINLOG_ADMIN,BINLOG_ENCRYPTION_ADMIN,CLONE_ADMIN,CONNECTION_ADMIN,ENCRYPTION_KEY_ADMIN,GROUP_REPLICATION_ADMIN,INNODB_REDO_LOG_ARCHIVE,PERSIST_RO_VARIABLES_ADMIN,REPLICATION_SLAVE_ADMIN,RESOURCE_GROUP_ADMIN,RESOURCE_GROUP_USER,ROLE_ADMIN,SERVICE_CONNECTION_ADMIN,SESSION_VARIABLES_ADMIN,SET_USER_ID,SYSTEM_USER,SYSTEM_VARIABLES_ADMIN,TABLE_ENCRYPTION_ADMIN,XA_RECOVER_ADMIN ON *.* TO `ambari`@`msl-dpe-perf80-100g.msl.lab` |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.01 sec)
... View more
08-21-2019
04:58 PM
yes. I posted with a link to ambari installation document. Like this: https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/set_up_the_ambari_server.html could this be issue? btw, one of my question is showing up now.
... View more
08-21-2019
04:19 PM
Hello, I had trouble to post questions on the community. It keeps marking my question as spam and I dispute to moderator get no answers. Does anyone know how to post questions without being treated as spam thanks, Harry
... View more
08-21-2019
03:54 PM
I am trying to install HDP 3.1 with Ambari 2.7 following this document https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/set_up_the_ambari_server.html All software installation completed and mysql database had been created. I then setup ambari with "ambari setup" followed by "ambari start". But I got follwing errors. I had listed the screen output for ambari setup and part of ambari-server.log to show the error message. Please help. Thanks, Harry Setup and start ambari-server [root@msl-dpe-perf80-100g resources]# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost): msl-dpe-perf80-100g.msl.lab
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?
Extracting system views...
.....
Ambari repo file contains latest json url <a href="<a href="http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json" target="_blank">http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json</a>" target="_blank"><a href="http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json</a" target="_blank">http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json</a</a>>, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
[root@msl-dpe-perf80-100g resources]# ambari-server start
Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start........................................ERROR: Exiting with exit code -1.
REASON: Ambari Server java process has stopped. Please check the logs for more information.
[root@msl-dpe-perf80-100g resources]# view /var/log/ambari-server/ambari-server.log ambari-server.log errors: 2019-08-21 15:30:03,832 INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.UpgradeItemEventCreator
2019-08-21 15:30:03,832 INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.UserEventCreator
2019-08-21 15:30:03,832 INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ValidationIgnoreEventCreator
2019-08-21 15:30:03,832 INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ViewInstanceEventCreator
2019-08-21 15:30:03,832 INFO [main] AuditLoggerModule:82 - Binding audit event creator class org.apache.ambari.server.audit.request.eventcreator.ViewPrivilegeEventCreator
2019-08-21 15:30:04,856 INFO [main] HostRoleCommandDAO:277 - Host role command status summary cache enabled !
2019-08-21 15:30:04,858 INFO [main] TransactionalLock$LockArea:121 - LockArea HRC_STATUS_CACHE is enabled
2019-08-21 15:30:05,081 INFO [main] AmbariServerConfigurationProvider:68 - Registered org.apache.ambari.server.ldap.service.AmbariLdapConfigurationProvider in event publisher
2019-08-21 15:30:05,584 INFO [main] CredentialStoreServiceImpl:60 - Initialized the temporary credential store. KeyStore entries will be retained for 90 minutes and will be actively purged
2019-08-21 15:30:05,593 INFO [main] LockFactory:54 - Lock profiling is disabled
2019-08-21 15:30:05,594 INFO [main] AmbariServer:1083 - Getting the controller
2019-08-21 15:30:06,282 INFO [pool-5-thread-1] AmbariServerConfigurationProvider:98 - JPA initialized event received: JpaInitializedEvent{eventType=JPA_INITIALIZED}
2019-08-21 15:30:06,282 INFO [pool-5-thread-1] AmbariServerConfigurationProvider:122 - Loading ldap-configuration configuration data
2019-08-21 15:30:06,377 INFO [main] AbstractPoolBackedDataSource:212 - Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 5, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 2vsgnka41ekyetann3ejj|57bc27f5, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> com.mysql.jdbc.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 2vsgnka41ekyetann3ejj|57bc27f5, idleConnectionTestPeriod -> 7200, initialPoolSize -> 5, jdbcUrl -> jdbc:mysql://msl-dpe-perf80-100g.msl.lab:3306/ambari, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 14400, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 32, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> SELECT 1, privilegeSpawnedThreads -> false, properties -> {user=******, password=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
2019-08-21 15:30:36,719 WARN [C3P0PooledConnectionPoolManager[identityToken->2vsgnka41ekyetann3ejj|57bc27f5]-HelperThread-#1] BasicResourcePool:223 - com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@30120b4 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
at sun.reflect.GeneratedConstructorAccessor48.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2570)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2306)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:839)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
at sun.reflect.GeneratedConstructorAccessor42.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:421)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:350)
... View more
Labels:
- Labels:
-
Apache Ambari
02-04-2019
03:50 PM
Hi Geoffrey, I have 4 data nodes and no snapshots set. Here are the output from the commands hdfs@msl-dpe-perf88:/$ hdfs dfs -df -h
Filesystem Size Used Available Use%
hdfs://msl-dpe-perf88.msl.lab:8020 28.2 T 27.1 T 0 96%
hdfs@msl-dpe-perf88:/$ hdfs lsSnapshottableDir
hdfs@msl-dpe-perf88:/$
... View more
02-02-2019
12:28 AM
Hi Geoffrey, I had been using -skipTrash options when I deleting files and /user/hdfs/.Trash directory is empty. I had also used -expunge command 24 hours ago. I still did not see disk space being freed. Here is results from dfsadm command hdfs@msl-dpe-perf88:/$ hdfs dfs -ls /user/hdfs/.Trash
hdfs@msl-dpe-perf88:/$
hdfs@msl-dpe-perf88:/$ hdfs dfsadmin -report
Configured Capacity: 31048107810816 (28.24 TB)
Present Capacity: 29767722012672 (27.07 TB)
DFS Remaining: 0 (0 B)
DFS Used: 29767722012672 (27.07 TB)
DFS Used%: 100.00%
Under replicated blocks: 97449
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
... View more
02-01-2019
11:25 PM
My HDFS has total disk space of 28.2 TB, which I have 15.1TB useful data on it. After a while, Ambari reports the disk space is 75% full, so I started "Balance HDFS" from Ambari. Since then, the available disk space decrease slowly until they are all gone. Now I have no more useful disk space. How can I reclaim the unused disk space. hdfs@msl-dpe-perf88:/$ hdfs dfs -du -h -s /
15.1 T /
hdfs@msl-dpe-perf88:/$ hdfs dfs -df -h
Filesystem Size Used Available Use%
hdfs://msl-dpe-perf88.msl.lab:8020 28.2 T 27.1 T 0 96%
... View more
Labels:
- Labels:
-
Apache Hadoop
10-17-2018
11:00 PM
I need to expand an existing cluster with additional data node. This question had been answered earlier, but the document link is broken. Can anyone resend the link or help answer the question again?
... View more
09-21-2018
06:18 PM
Yes. And I accepted the answer. Thanks.
... View more