I am a hadoop-newbie and I have a hard time installing a hadoop single node cluster using ambari 2.2.2.
I am trying to install on a dual core machine with 2 GB of RAM running CentOS 6.7 (a real machine - no VM).
I installed by following each single step of this howto:
I installed the following services (all on the same machine as I do not have other machines available):
HDFS, MaReduce2 Yarn Tez Hive HBase Pig Zookeeper Ambari-Metrics SmartSense
After the installation has completed I get several warnings that several services could not be started.
The machine seems to run on heavy load and there are huge lags after any action which makes troubleshooting extremely hard.
The Alert page looks as follows:
Can anyone provide some hints on how to resolve this?
Thank you in advance,
Comment: Maybe I have made a mistake setting up the hostname. My /etc/hosts file contains:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.178.77 HadoopNode1.ts.de HadoopNode1
The network adapter is configured to use the static IP address above.
I configured ambari to use "HadoopNode1.ts.de" as the only machine in the cluster.
I diuble checked that ssh works password-less for this machine. If I login to the machine and submit the command "ssh HadoopNode1.ts.de" then I am logged in via ssh immediately. So this definately works.
When check CPU usage with the "top" command I see several "java" processes and one "python" process which produce high CPU load.
@Anonym User It might help to include one of the error logs from a service to get more details. Please include the HDFS log since this is the first service to try to start and if it fails the other services will fail.
My suggestion is to thoroughly review the install pre-reqs. In particular, make sure you set the ulimit to a large amount. I've found setting ulimit to 100000 works. http://docs.hortonworks.com/HDPDocuments/Ambari-18.104.22.168/bk_Installing_HDP_AMB/content/_check_the_max...
Thank you Scott for your very quick reply.
Setting the ulimit to 100.000 persistently (!) has solved some of my problems but not all of them.
The installation manual for ambari (which I did follow thoroughly) suggests to set the ulimit value (which I did) but it does NOT say that the proposed commands do not change the limit persistently! So after finding out how to make it persistent the machine is a lot more responsive now.
The reason why I installed Hadoop on a separate machine instead of using the sandbx-VM was that it would not run in a VM reliably. I found out that the ulimit setting in the Sandbox VM is also too low and that was the reason why my VM installation was very unresponsive and unreliable. The VM is fixed now and working well.
I wonder why this it not set properly by HortonWorks and should not every user run into this problem?
The native installation is still not working. Is it possible that it can't run with only 2 GB of memory?
If it should run with 2GB then I guess I will have to dive into the logfiles and try to find out what the problem is.