Member since
01-19-2017
3679
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 908 | 06-04-2025 11:36 PM | |
| 1509 | 03-23-2025 05:23 AM | |
| 744 | 03-17-2025 10:18 AM | |
| 2680 | 03-05-2025 01:34 PM | |
| 1786 | 03-03-2025 01:09 PM |
05-31-2018
10:39 PM
1 Kudo
@FELIPE CABRAL Your /etc/hosts should look like this in the below example I used 192.168.0.124 as the IP of edcserver.domain please use the appropriate IP The first 2 lines shouldn't be tampered with # cat /etc/hosts
127.0.0.1 localhost localhost.localdomain
::1 localhost localhost.localdomain
192.168.0.124 edcserver.domain edcserver Now to overcome the error 1 other registered host (screenshot 2018-05-31-18-09-36-window.png) you have to delete this entry from 2 tables hoststate and hosts in the Ambari database. I have attached the procedure below, the assumption is as follows Ambari database name = ambari Ambari database type= MySQL Ambari database password =Secret1234 If you are using postgres please adapt accordingly the SQL Log on the DB # mysql -u root -pSecret1234 some output here Check the databases mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| ambari |
| hive |
| mysql |
| oozie |
| ranger |
| rangerkms |
| test |
+--------------------+
8 rows in set (0.00 sec) Connect to the Ambari database mysql> use ambari;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed Check the structure if hoststate table we need to delete the host_id mysql> describe hoststate;
+-------------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+---------+-------+
| agent_version | varchar(255) | NO | | NULL | |
| available_mem | bigint(20) | NO | | NULL | |
| current_state | varchar(255) | NO | | NULL | |
| health_status | varchar(255) | YES | | NULL | |
| host_id | bigint(20) | NO | PRI | NULL | |
| time_in_state | bigint(20) | NO | | NULL | |
| maintenance_state | varchar(512) | YES | | NULL | |
+-------------------+--------------+------+-----+---------+-------+
7 rows in set (0.03 sec)
mysql> select host_id from hoststate;
+---------+
| host_id |
+---------+
| 1 |
+---------+
1 row in set (0.04 sec)
mysql> Delete from hoststate where host_id=1;
Query OK, 1 rows affected (0.00 sec) Check the hosts table mysql> describe hosts;
+------------------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+---------------+------+-----+---------+-------+
| host_id | bigint(20) | NO | PRI | NULL | |
| host_name | varchar(255) | NO | UNI | NULL | |
| cpu_count | int(11) | NO | | NULL | |
| cpu_info | varchar(255) | NO | | NULL | |
| discovery_status | varchar(2000) | NO | | NULL | |
| host_attributes | longtext | NO | | NULL | |
| ipv4 | varchar(255) | YES | | NULL | |
| ipv6 | varchar(255) | YES | | NULL | |
| last_registration_time | bigint(20) | NO | | NULL | |
| os_arch | varchar(255) | NO | | NULL | |
| os_info | varchar(1000) | NO | | NULL | |
| os_type | varchar(255) | NO | | NULL | |
| ph_cpu_count | int(11) | YES | | NULL | |
| public_host_name | varchar(255) | YES | | NULL | |
| rack_info | varchar(255) | NO | | NULL | |
| total_mem | bigint(20) | NO | | NULL | |
+------------------------+---------------+------+-----+---------+-------+
16 rows in set (0.02 sec) Retrieve and delete the host with the matching id 1 mysql> select host_id from hosts;
+---------+
| host_id |
+---------+
| 1 |
+---------+
1 row in set (0.00 sec)
mysql> Delete from hoststate where host_id=1;
Query OK, 1 rows affected (0.00 sec) And restart ambari-server and proceed with the registration all should be fine. Please revert If you found this answer addressed your question, please take a moment to log
in and click the "accept" link on the answer.
... View more
05-31-2018
09:08 PM
@Ian Bradshaw The Ambari server logs ambari-server.log are in /var/log/ambari-server/* you could use WinSCP to copy them to your localhost and attach it to this thread.
... View more
05-31-2018
07:39 PM
@FELIPE CABRAL Can you share the entry of your /etc/hosts on the Ambari server? Select Version Can you remove the repositories that doesn't match your node either the Redppc7 or Redhat Install Option Please use the FQDN rather than localhost, see the output of below command # hostname -f Please revert
... View more
05-31-2018
05:12 PM
@raghavendra v I understand but but rollback to the previous version you will need the GUI to swtch between the GRB version you could install and uninstall there after ? If not then ask you SYSOPS team to do that 🙂
... View more
05-31-2018
01:30 PM
@raghavendra v Do you have GUI on the servers? If not can you install because you will have to boot from the old kernel and backup the directory where your KMS keys?
... View more
05-31-2018
01:07 PM
@raghavendra v To be able to help you need to share HDP version OS type and versions Cluster type (single/multi-node) Cluster Size(affected nodes) Just an important info you have.
... View more
05-31-2018
12:41 PM
@Mohamed Farook Here is the procedure to switch Ambari servers on an existing HDP cluster
... View more
05-31-2018
08:00 AM
@Samant Thakur Did you go through the links I posted above? Your data node and Namenode heap sizes need some tuning. Are you seeing any data node high HEAP SIZE alert? Estimating NameNode Heap Memory Needed Memory is estimated by considering the capacity of a cluster. Values are rounded. The below cluster physically stores 4800 TB, or approximately 36 million block files (at the default block size). Replication determines how many namespace blocks represent these block files. At capacity, with the recommended allocation of 1 GB of memory per million blocks, The Cluster needs 12 GB of maximum heap space. 200 hosts of 24 TB each = 4800 TB. Blocksize=128 MB, Replication=3 Cluster capacity in MB: 200 * 24,000,000 MB = 4,800,000,000 MB (4800 TB) Disk space needed per block: 128 MB per block * 3 = 384 MB storage per block Cluster capacity in blocks: 4,800,000,000 MB / 384 MB = 12,000,000 blocks At capacity, with the recommended allocation of 1 GB of memory per million blocks, This cluster will need 12 GB of maximum heap space. Hope that helps please revert!
... View more
05-30-2018
08:27 PM
@Samant Thakur There is definitely a network problem with the 3 nodes are they same hardware? NIC and network? Average IO rate mb/sec: 27.063095092773438
Average IO rate mb/sec: 19.786481857299805 "Allocation Failure" is a cause of GC cycle to kick. A GC allocation failure means that the garbage collector could not move objects from young gen to old gen fast enough because it does not have enough memory in old gen. This can cause application slowness. Whats your DataNode maximum Java heap size? https://community.hortonworks.com/questions/64677/datanode-heapsize-computation.html https://community.hortonworks.com/questions/45381/do-i-need-to-tune-java-heap-size.html https://community.hortonworks.com/questions/78981/data-node-heap-size-warning.html Do you have a NameNode HA component? if so it may be that failover has occurred -but the client doesn't detect this and retry its operation. If it isn't a production cluster can you restart all the components?
... View more
05-30-2018
07:38 PM
@Rodrigo Hjort The files look okay except the krb5.conf notice the entry [domain_realm] please change it on the kdc and copy it to all the nodes and restart the KDC services(kadmin & Krb5kdc), such a simple thing could be the problem [libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = MYHDP.COM.BR
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
default_ccache_name = /tmp/krb5cc_%{uid}
#default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
#default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
[domain_realm]
.myhdp.com.br = MYHDP.COM.BR
myhdp.com.br = MYHDP.COM.BR
[logging]
default = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
kdc = FILE:/var/log/krb5kdc.log
[realms]
MYHDP.COM.BR = {
admin_server = 10.30.242.46
kdc = 10.30.242.46
} How is your /etc/hosts entry? No mixed cases Why is Ranger,Yarn,MR ,AMS disabled/maintenance mode? Had you enabled ranger plugins? Please revert
... View more