About robi

robi · ‎06-06-2017

Since the connection refused error stopped me everytime, I decided to install ambari.repo and yum install ambari-agent on each node in cluster. so install cluster, 3 hosts failed at the first step--confirm hosts. only ambari-server host passed it. the error was, failed to reconnect to local server as above. so seemed ip on the node have changed. I may have to use the VCP to have static IP, right? if yes, the limit for VCP is 5, I have over 10 nodes here. Also, the host check has no warnings... Please help. thanks,

robi · ‎06-05-2017

I have installed HDP many times, always have an issue-- connection failed and connection refused when start ambari metrics. 1. during ambari install, I have ignored the ams_env warning msg. if I follow the recommended number, it will come up with newer number, I kept change for many time, one time I even changed 7 times, still not get it pass. so I ignored this warning, should I? 2. all ambari-metrics-collcter and ambari-metrics-monitor are installed, I can see from rpm -qa|grep ambari-collecter 3. finally, ambari-metrics just dont start, get all connection failed error. so seemed the configuration cause the connection failed, so any one have good combination to edit ams_env and ams_habase_env? thank you very much.

robi · ‎06-05-2017

oh, my ambari service is external mode, use http://ec2....amazon.com:8080, if this is so called external mode. after ambari-server reset, I got the same error. the repo is: http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo rpm -qa|grep ambari-metrics -- show all packages are installed. so the hbase configure info is the problem. it refuse the connection. I dont need to edit hbase configure with ambari 2.4 in the past at all. any changes on HDP? thanks, Robin

robi · ‎06-05-2017

oh, my ambari service is external mode, use http://ec2....amazon.com:8080, if this is so called external mode. after ambari-server reset, I got the same error. the repo is: http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo rpm -qa|grep ambari-metrics -- show all packages are installed. so the hbase configure info is the problem. it refuse the connection. I dont need to edit hbase configure with ambari 2.4 in the past at all. any changes on HDP? thanks, Robin

robi · ‎06-05-2017

oh, my ambari service is external mode, use http://ec2....amazon.com:8080, if this is so called external mode. after ambari-server reset, I got the same error. the repo is: http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo rpm -qa|grep ambari-metrics -- show all packages are installed. so the hbase configure info is the problem. it refuse the connection. I dont need to edit hbase configure with ambari 2.4 in the past at all. any changes on HDP? thanks, Robin

robi · ‎06-05-2017

oh, my ambari service is external mode, use http://ec2....amazon.com:8080, if this is so called external mode. after ambari-server reset, I got the same error. the repo is: http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo rpm -qa|grep ambari-metrics -- show all packages are installed. so the hbase configure info is the problem. it refuse the connection. I dont need to edit hbase configure with ambari 2.4 in the past at all. any changes on HDP? thanks, Robin

robi · ‎06-05-2017

on my ambari server and datanode, both has no ambari-metrics-collector found. [root@ip-172-31-1-92 etc]# ls -lart /etc/ambari-metrics-collector ls: cannot access /etc/ambari-metrics-collector: No such file or directory [root@ip-172-31-12-243 etc]# ls -lart /etc/ambari-metrics-collector ls: cannot access /etc/ambari-metrics-collector: No such file or directory 1. when add volume 100gb or 800gb, I still have the same connection problem 2. this time I didnt add new volume to EBS, only 20gb for root /. 3. I reinstall more than 10 time, always the same issue here. so do you think my security settings is the problem, here is my security group info, all TCP, HTTP all ports, all UDP and all traffice with anywhere access. this is wide open security. Do you think the adding ambari id_rsa.pub to authorized_keys on all servers for passwdless is enough? From ambari server, I can passweless ssh to all other servers including itself, do you think I need to make passwdless between all nodes? thank you for your help.

robi · ‎06-05-2017

605-hdp-error.txt on one of the datanode. no ambari-metrics-collector.log on ambari server. this is metrics log file. I only have 20g for root dir /, no EBS storage assigned here

robi · ‎06-05-2017

you are right, Jay. the problem is port connection refused. however, my /tmp is 777, so what to do from here. [root@ip-172-31-12-243 /]# free -m total used free shared buffers cached Mem: 16077 390 15687 3 81 74 -/+ buffers/cache: 234 15843 Swap: 0 0 0 [root@ip-172-31-12-243 /]# hostname -f ip-172-31-12-243.ca-central-1.compute.internal telnet ip-172-31-1-92.ca-central-1.compute.internal 3000 Trying 52.60.67.105... telnet: connect to address 52.60.67.105: Connection refused [root@ip-172-31-12-243 /]# telnet ip-172-31-1-92.ca-central-1.compute.internal 16000 Trying 52.60.67.105... telnet: connect to address 52.60.67.105: Connection refused thank you so much for helping here.

robi · ‎06-05-2017

I only can upload one of my error file. but most of them are the same, connection failed...and seemed ambari-metrics never can be started. Only ZK is up. so please help. thank you very much.

robi · ‎06-05-2017

I used the following to setup these recommended size I found in this community help ams-env :: collector_heapsize = 2048 ams-hbase-env :: hbase_master_heapsize = 512 ams-hbase-env :: hbase_master_xmn = 102 ams-hbase-env :: hbase_regionserver_heapsize = 4096 ams-hbase-env :: regionserver_xmn_size = 512 604-hdp-errors.png

robi · ‎06-05-2017

Thank you so much, let me try it and get back to you in couple of hours.

robi · ‎06-05-2017

Amabari installation failed, the log file -->Installing package ambari-metrics-collector ('/usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector') command failed on my ambari server, I dont have this package at all, anyhow, I copied HDP.repo, HDP-UTILS.repo to all data nodes then try to manually install ambari-metrics-collector, but I got following: /usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector Error: Nothing to do /usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector Error: Nothing to do these errors repeatedly stop my installation. and my repo was downloaded as: 1. I used this repo -->http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo 2. I used AWS/EC2 m4.large with 20 gb for Root dir and 100 EBS(magnetic), I ignored the metrics_collector_heap_size warnings, because if I following the recommended size to modify them, it would come up with the second recommended size, I even go as far as 7 times edit these numbers, never done with it. 3. Any one had the same issue like mine? if the storage setting here matters? what is a good combination of these metrics_collector_heap_size, xmn_size and master_heap_size? 4. the type of instance are different, different RAM, these heap_size will depends, what to follow? isnt this HDP job? thank so much for you help.

robi · ‎06-01-2017

you are right, Since my setting all working before I never use VCP and just thought about this is may be a cause. I will work on it and get back to you. thanks again.

robi · ‎06-01-2017

Thank you so much for this quick respond. why are IP and hostname changed? my network.hostname=localhost, no change on /etc/sysconfig/network file. did I missed anything? is this AWS/EC2 networking DNS problem? I already created a ticket to AWS support. do you think I can fix it on my side.

robi · ‎06-01-2017

All my nodes and services have this same error --Connection failed to http://ip-172-31-18-1.ca-central-1.comp...'. I believe I setup passwdless correctly(I done it many times and all worked before), my security group to setup all TCP, all UDP and SSH. do any one think this is AWS/EC2/DNS problem? since all connection fails, no exception. any idea? thank you so much for your help.

robi · ‎05-31-2017

thanks, I got your another post and tried ambari 2.2 so dont need to deal with smartsense.

robi · ‎05-31-2017

still like to know the best combination of RHEL version and ambari.repo version. document says it will fit all, but different error comeup.

robi · ‎05-31-2017

yes, the smartsense error is not showing any more. thank you so much for that.

robi · ‎05-31-2017

thank you for your reply. 1. I tried all conbination of AWS/EC2 RHEL 6. 7 and ambari 2.4, 2.2. I can not avoid smartsense error, also, the ambari.repo download have no smartsense included 2. in order to download smartsense, the account to hortonworks.support.com is needed, have no idea to create a new account on it 3. I need a combination of RHEL version and ambari version and HDP version from you in order not deal with smartsense, if you have one. Meanwhile, I will try ambari.repo 2.2. hope I will not have issues with smartsense. your help is greatly appreciated. best answer to you.

robi · ‎05-31-2017

when install ambari , the error repeatedly asking for smartsense package, it is not in ambari.repo, and only can be downloaded from hortonworks.support.com, and it need account info. I like to know how to create a new user account at this website.

robi · ‎05-31-2017

BTW, we dont have HDP support here. we just test it for a project.

robi · ‎05-31-2017

thank you for this info, however, it says 'For new installs, SmartSense will be installed by default. Entering your Hortonworks SmartSense ID is still required to activate SmartSense. No bundles will be captured and sent until the a valid SmartSense ID is entered.' I tried ambari 2.5, it asked SmartSense ID and provided my company email. I got nothing so far. so without smartsense ID, you never can install HDP, right?

robi · ‎05-31-2017

I have no account to log onto https://support.hortonworks.com. how to get this smarsense package then?

robi · ‎05-22-2017

According to the mongodb doc here, https://www.mongodb.com/blog/post/in-the-loop-with-hadoop-take-advantage-of-data-locality-with-mongo-hadoop This means that we can accomplish local reads by putting a DataNode process (for Hadoop) or a Spark executor on the same node with a MongoDB shard and a mongos. In other words, we have an equal number of MongoDB shards and Hadoop or Spark worker nodes. We provide the addresses to all these mongos instances to the connector, so that the connector will automatically tag each InputSplit with the address of the mongos that lives together with the shard that holds the data for the split. the above mentioned that we have an equal number of MongoDB shards and Hadoop or Spark worker nodes. I think it means number of mongodb shard = data nodes with spark. so my next step is to ensure sparks installed on all data nodes and then mongodb shards on these worker nodes. Let me know if you think differently. thanks Robin

robi · ‎05-22-2017

I installed HDP cluster with 4 data nodes. and add service mongodb/mongos/mongodconfig, which is mongodb with sharding I tried to add them to the HDP data nodes (2 grp and 2 nodes each). Somehow, the mongodb and mongos are unable to start. so I start wondering, 1. if HDP, mongodb/mongos, should we still need data nodes in the system? 2. datanode and mongodb sharding nodes can be on the same node? 3. if not, Can we have 0 datanode in HDP cluster? 4. the kafka is in the system, we need to use kafka/spark, so data nodes are needed for spark/scala RDD in this architecture, right? Any one have any clue on this, please help. thank you very much. Robin

robi · ‎05-19-2017

I will close this case I got my answer here.

robi · ‎05-19-2017

yes, this is the piece of info can fix my problem here. thank you so much Eyad Gareinabi. Robin

robi · ‎05-19-2017

Anyone have info on how mongodb do the sharding within the HDP cluster? I have a mongodb installed by ambari, but when check the sharding info on mongodb, it tells me the sharding is not enabled. so when I tried to enable sharding by commands, mongodb server will be down. I have to restart all components on that server.

robi · ‎05-19-2017

Anyone have info on how mongodb do the sharding within the HDP cluster? I have a mongodb installed by ambari, but when check the sharding info on mongodb, it tells me the sharding is not enabled. so when I tried to enable sharding for mongodb, the server will be down. I have to restart all components on that server.

Online	Offline
Last Visited	‎11-12-2019 02:14 AM

Date Registered	‎08-28-2015 01:05 AM
Last Visited	‎11-12-2019 02:14 AM
Total Messages Posted	194
Total Kudos Received	45

Re: Ambari install failed

ERROR Unable to reconnect to https://ip-172-31-13-...

need info for ams_hbase_env and ams_env configurat...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Re: Error: No package ambari-metrics-collector ava...

Error: No package ambari-metrics-collector availab...

Re: AWS/EC2 ambari installation error 'Connection ...

Re: AWS/EC2 ambari installation error 'Connection ...

AWS/EC2 ambari installation error 'Connection fail...

Re: I got this error 'No package smartsense-hst av...

Re: need info how to create a new account at https...

Re: need info how to create a new account at https...

Re: need info how to create a new account at https...

need info how to create a new account at https://s...

Re: I got this error 'No package smartsense-hst av...

Re: I got this error 'No package smartsense-hst av...

I got this error 'No package smartsense-hst availa...

Re: if set up mongodb sharding with HDP cluster, t...

if set up mongodb sharding with HDP cluster, the d...

Re: Anyone have info on how mongodb do the shardin...

Re: Anyone have info on how mongodb do the shardi...

Anyone have info on how mongodb do the sharding wi...

Anyone have info on how mongodb do the sharding w...