Thank you Cdraper, Finally I use this combination, RHEL 7.3 (m4.large) , ambari 2.4.10 and HDP 2.5 install 7 nodes on AWS. the only issues I deal with is change the namenode java heap size to 4gb in my case. then everything worked. your info is still very helpful. I am going to setup kafka 3 nodes and mongodb from here. will keep you posted. thanks again. Robin ... View more

thank you for getting back to me. 1 iptables -L dont have any ouput 2. the firewall package is available but not installed in a AWS/EC2 instance. therefore, I did run systemctl stop firewalld and systemctl disable firewalld on REHL 7 and service iptables stop, chkconfig iptables off on REHL6. 3. the point here, if these firewall or iptables package is not installed, I may have to think they are never act, so if not installed, they are not there = disabled. let me know if you think differently. thanks, ... View more

Thank you Cdraper for getting back to me. 1. I dont have HA setup, but I think HDP requires to have secondary name node, I didnt do any configure yet, but snamenode is on the list to start 2. do yo think Hbase space issue could cause this problem. as Hbase stores all metadata for namenode and cluster, if Hbase has problem, then namenode cant start, so does snamenode then HDFS then entire cluster? 3. if I use m4.xlarge instance on AWS, ended up with warnings on hbase_master_heapsize, hbase_collector_heapszie etc, if you change to the recommended values, the new sets of parameters come up. one time I have changed 7 times and never get rid of that warning. if I ignore the warning, my namenode never, never started. 4. if I use m4.large, I dont get the these heapsize warnings, instead I get warning on packages not install, so I tried to manually install them on host, it just dont execute or yum dont have this package. I am kind of give up on HDP 2.4 install on RHEL 6, 7/AWS/EC2, so what do you suggest, shall I try ubuntu or Centos, which version? HDP stable version is 2.4.1.0 and ubuntu 12? After all these experiment, I just feel HDP with AWS is not compatible, I could be wrong. either AWS/EC2 root / size too samll, you can resize in instance creation process, but dont get the size you define. if instance RAM = 8gb or more, then HDP get lost on configure them, that is why all these heap_size size issues.... sorry long story, thank you for your help. Robin ... View more

Clink, I got the same error, and I changed my /etc/hosts file as: 52.60.239.41 ip-172-31-12-19.ca-central-1.compute.internal node1 also, I restart agent on each hosts, 4 of them, I still get the same error. please help.   Robin ... View more

Clint, my hosts file look like this, and seemed it is not working. 52.60.239.41 ip-172-31-12-19.ca-central-1.compute.internal 52.60.157.176 ip-172-31-3-218.ca-central-1.compute.internal 35.182.18.189 ip-172-31-3-179.ca-central-1.compute.internal 35.182.25.52 ip-172-31-2-137.ca-central-1.compute.internal do I have to give a short name at end, where this shortname come from my EC2 instance name, right? ... View more

thank you so much. even I need more time to digest it, I can tell this is what I need to know. thank you for spending time to help. Robin ... View more

oh, my ambari service is external mode, use http://ec2....amazon.com:8080, if this is so called external mode. after ambari-server reset, I got the same error. the repo is: http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo rpm -qa|grep ambari-metrics -- show all packages are installed. so the hbase configure info is the problem. it refuse the connection. I dont need to edit hbase configure with ambari 2.4 in the past at all. any changes on HDP? thanks, Robin ... View more