Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

abhishes — Fri, 16 Sep 2022 09:02:50 GMT

I have one laptop with 16 GB RAM and on this I have setup 3 centos VMs (virtual box) with 3 GB RAM Each.

my laptop is behind a WiFi network and has static IP. I have configured networking in the VM using "bridged network" and each VM has dedicated IP and each VM can ping each other, it can ping the HOST machine and each VM can connect to the internet.

I have disabled the firewall on each vm using (systemctl disable firewalld.service).

I installed all the software for CDH 4.1

systemctl disable firewalld.service

reboot

chmod 755 jdk-6u45-linux-x64-rpm.bin
./jdk-6u45-linux-x64-rpm.bin

wget http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/cloudera-cdh4.repo

cp cloudera-cdh4.repo /etc/yum.repos.d/

rpm --import http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/RPMGPG-KEY-cloudera

yum install hadoop-hdfs-namenode

yum install hadoop-hdfs-journalnode

yum install zookeeper-server

yum install hadoop-hdfs-zkfc

service zookeeper-server init --myid=1

yum install zookeeper-server

service zookeeper-server start

I edited the file /etc/zookeeper/conf/zoo.cfg

maxClientCnxns=50
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
server.1=192.168.1.30:2888:3888
server.2=192.168.1.31:2888:3888
server.3=192.168.1.32:2888:3888

but when I start the service zookeeper-server start. I can see too big problems

when I do nmap -sT -O 192.168.1.30

I only see

22/tcp open ssh.

So my question is that why is there no listener on any port for zookeeper-server

If I try to do service zookeeper-server start then it says the process is already running.

The second problem is that the zookeeper.log says

2014-07-18 17:56:29,910 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@368] - Cannot open channel to 2 at election address nn2.abhishek.com/192.168.1.31:3888
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:354)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:327)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365)
at java.lang.Thread.run(Thread.java:662)
2014-07-18 17:56:29,911 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@368] - Cannot open channel to 3 at election address jt1.abhishek.com/192.168.1.32:3888
java.net.NoRouteToHostException: No route to host
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:354)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:327)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365)
at java.lang.Thread.run(Thread.java:662)

I can assure you that I have typed this command on every VM

systemctl disable firewalld.service

systemctl stop firewalld.service

Please help me here I am stuck with this for almost a week now.

I found this blog which has some details of the problem faced by me

http://www.cyberciti.biz/tips/no-route-to-host-error-and-solution.html

1) Can you ping your router

yes I can ping 192.168.1.1

2) Is firewall blocking you.

As you can see I have disabled the firewall.

3) Finally make sure you are using a router and not a proxy server. Proxy servers are good for Internet browsing but not for other work such as ftp, sending ICMP request and so on.

not sure about this one. it is a WiFi router.

Please help me here. is there anyone over here who has tried something similar? to have 2 VMs on a single laptop? what was the outcome?

Re: Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

abhishes — Mon, 21 Jul 2014 13:57:01 GMT

Can someone please help me out here.

If you have a laptop connected to a wifi network. please create 2 VMs and try to start the zookeeper server.

This has not been working for me as zookeeper will just not start. Can you please help.

Re: Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

ben.hemphill — Mon, 21 Jul 2014 21:46:54 GMT

It's certainly possible, presuming you have enough ram and cpu to run a full stack of JVMs.

have you verified that you have a myid file for each vm?

the error output looks like it is from 192.168.1.30, correct?

from that it looks like you actually got to 192.168.1.31:3888 but it wasn't listening, it also looks like something is blocking 192.168.1.32:3888 since you got a no route to host exception.

Re: Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

abhishes — Sat, 26 Jul 2014 07:47:19 GMT

Yes your suggestion is right. the 32 machine did not have the firewall switched off.

when I did serivce stop firewalld.server and service disable firewalld.service it started to work fine.

question Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop in Archives of Support Questions (Read Only)

Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

Re: Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

Re: Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop

Re: Is it possible to have a 3 VM CDH 4.1 cluster on 1 laptop