Thanks Neeraj for all these details. I was looking for a way to configure this, you provide exact steps. thank you. Robin ... View more

I tried to upload file from windows to linux, so I use pscp -scp option, it connected, but failed to copy. thanks, ... View more

thank you, let me know it now. ... View more

yes, got it. thank you so much. ... View more

thank you Neeraj. so I need to download and upload this file to my server, it would be all file I need, right? wget http://public-repo-1.hortonworks.com/HDP/tools/2.3.4.0/hdp_manual_install_rpm_helper_files-2.3.4.0.3485.tar.gz tar zxvf hdp_manual_install_rpm_helper_files-2.3.4.0.3485.tar.gz ... View more

thank you Neeraj, so any user can log on do anything with Hortonworks without su - xxx? ... View more

OH, yest, you are right. I was little confused. ... View more

this is the best answer for my question. ... View more

so this one I picked would work, right? jdk-8u71-linux-x64.gz @http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html Appreciated your valuable input, never thought this is a custom JDK. ... View more

I made all other changes ok. ... View more

to disable THP if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag fi then I got > dont know what to do next. ... View more

@Ancil McBarnett Great, appreciated your quick respond. we will try this tool to see what comes out. Thanks again. Robin ... View more

yes, both answers provided me the best practice. I really appreciated. we are having a meeting on Monday to decide which is the best ETL tool for load data from taradata database to hortonworks cluster, so any suggestions. Sparks or anything else. I guess this is a new question, should I create another thread? ... View more

@Kuldeep Kulkarni and @Neeraj Sabharwal, Hi Gurus, you two provided all info I am looking for. thank you so much for your help. Seemed this community is a great place to get help and so many gurus out there... Thanks again. Robin ... View more

Thank you so much Ben for your help. u 1. I reinstalled EC2 2 ubunt instances with  m3.large type.     .iptables disabled by sudo ufw disable     . passwdless works in between instance     . modified /etc/sysctl.conf to disable ipv6 as well as in /etc/hosts     . download cloudera manager 5.2. my cluster is 2 nodes on EC2 unbuntu 12.04 LTS     . my hosts file look like following:     127.0.0.1 localhost     52.88.149.30 ec2-52-88-149-30.us-west-2.compute.amazonaws.com 172.31.33.55 ip-172-31-33-55     52.88.152.243 ec2-52-88-152-243.us-west-2.compute.amazonaws.com 172.31.33.56 ip-172-31-33-56   2. Install CM and never make it so far. alwasy failed to receive heartbeat from agent.   I have check the cloudera-scm-server and cloudera-scm-agent log file as you suggested. I copied and pasted the error part in these files as following   -----in the cloudera-scm-agent.log we got timeout [01/Sep/2015 22:36:41 +0000] 2532 HTTPServer Thread-2 _cplogging INFO [01/Sep/2015:22:36:41] ENGINE Stopped thread '_TimeoutMonitor'. [01/Sep/2015 22:36:41 +0000] 2532 HTTPServer Thread-2 _cplogging INFO [01/Sep/2015:22:36:41] ENGINE Bus STOPPED [01/Sep/2015 22:36:41 +0000] 2532 HTTPServer Thread-2 _cplogging INFO [01/Sep/2015:22:36:41] ENGINE Bus EXITING [01/Sep/2015 22:36:41 +0000] 2532 HTTPServer Thread-2 _cplogging INFO [01/Sep/2015:22:36:41] ENGINE Bus EXITED   ------in the supervisord.log, the part repleat many times in log file 2015-09-01 19:43:29,698 CRIT Supervisor running as root (no user in config file) 2015-09-01 19:43:29,734 INFO RPC interface 'supervisor' initialized 2015-09-01 19:43:29,735 INFO RPC interface 'supervisor' initialized 2015-09-01 19:43:29,738 INFO daemonizing the supervisord process 2015-09-01 19:43:29,739 INFO supervisord started with pid 7173 2015-09-01 19:43:30,746 INFO spawned: 'cmflistener' with pid 7174 2015-09-01 19:43:31,881 INFO success: cmflistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2015-09-01 19:49:01,744 INFO exited: cmflistener (terminated by SIGTERM; not expected) 2015-09-01 19:49:01,744 WARN received SIGTERM indicating exit request -------in the cmf-listener.log file [01/Sep/2015 22:19:11 +0000] 942 MainThread supervisor_listener INFO Starting event listener as pid 942 [01/Sep/2015 22:19:12 +0000] 942 MainThread supervisor_listener INFO Cannot open agent FIFO (agent probably dead), dropping event   I know everyone is very busy these days, I apprecite your precise time. please take a look and give me some clue to finish this cloudera manager installation.   thank you very very much.   Robin ... View more

HI Clint,   1. I am still installing Cloudera Manager 5 on EC2 ubuntu 12.04 LTS, but my /etc/sysconf/network file just dont have any tag like 'HOSTNAME='. Do I have to modify this file to fix the 'Failed to receive heartbeat from agent' ?   2. I notice, there is no way to disable selinux=disabled as I could not find /etc/sysctl.conf file on my ubuntu. I wonder because I security group set 'anywhere' for all access. please correct me if I am wong.   3. there is no way to check on iptables start/stop/status on these EC2 ubuntu instances, is this normal to ubuntu?   The doc of troubleshooting dont have much info from cloudera, I am very dispointed as I could not find any answer for my questions so far.   Could not find a simple sample of /etc/hosts file either. please help.   thanks,   Robin         ... View more

Any one can help with this issues? I am stuck by this issues, no matter how modify this /etc/hosts file, then reboot my instances, I always get the same error. I tried follwoing, it dont work.   127.0.0.1 localhost.localdomainlocalhost 52.88.118.48 ec2-52-88-118-48.us-west-2.compute.amazonaws.com ip-172-31-0-146.us-west-2.compute.internal 52.26.227.159 ec2-52-26-227-159.us-west-2.compute.amazonaws.com ip-172-31-0-147.us-west-2.compute.internal   or    127.0.0.1 localhost.localdomainlocalhost 52.88.118.48 ec2-52-88-118-48.us-west-2.compute.amazonaws.com ip-172-31-0-146 52.26.227.159 ec2-52-26-227-159.us-west-2.compute.amazonaws.com ip-172-31-0-147   or 127.0.0.1 localhost.localdomainlocalhost 52.88.118.48 ec2-52-88-118-48.us-west-2.compute.amazonaws.com  ec2-52-88-118-48  52.26.227.159 ec2-52-88-118-48 -159.us-west-2.compute.amazonaws.com  ec2-52-88-118-48   none of them worked, what is wrong with these hosts file??    Can anyone help?   thank you so much.   Robin ... View more