Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to Install Cloudera on Debian 8 (probably Ubuntu as well)

Highlighted

How to Install Cloudera on Debian 8 (probably Ubuntu as well)

New Contributor
I've followed these instructions for Debian Jessie, but they don't work:
 
 
This is how I got it up and running:

0) Install Oracle Java SDK, preferably with java-package or Zookeeper won't be able to find "java" command.
 
Install Debian Jessie plus Oracle Java SDK:
 
apt-get update && apt-get install java-package
Download Java 8 SE
Follow make-jpkg instructions.
 
1) Fix the repository file for Debian 8 (currently broken on Cloudera's site):
https://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh/cloudera.list <- Debian 8 wrong
 
Resulting file:
 
 
 
# Packages for Cloudera's Distribution for Hadoop, Version 5, on Debian 8.0 amd64
deb http://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh jessie-cdh5 contrib
deb-src http://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh jessie-cdh5 contrib
 
 
2) Install Maintainer's PGP Key 
 
wget https://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh/archive.key -O- | apt-key add -
 
Equivalent to Cloudera's documentation but cooler :-)
 
(sudo) apt-key adv  --keyserver pgp.mit.edu --recv 02A818DD
Even neater as it runs with just 1 command.
 
3) Update repository cache
  
apt-get update
 
4) Zookeeper Installation
 
apt-get install -t jessie-cdh5 zookeeper
 
I've forced it from Cloudera's repository. Both Debian and Cloudera provide the same version, but the OS prefers Debian's repository. I'd suggest adding this explicitly to Cloudera's docs or removing Zookeeper from CDH's repository if there's no compatibility issues. In the first case, it could make the system unstable and in the second, you'll never get that Zookeeper if you don't select it explicitly.
 
5) Start Zookeeper ---------> TROUBLESOME
 
Is it even required in a single server setup??? IMHO not, but I can't ensure it...
 
Fails to bring it up if you don't install java command under a standard PATH. That's why I've switched from Cloudera's suggested Java setup to Debian's java-package tool.

root@cloudera:~# /usr/lib/zookeeper/bin/zkServer.sh start
JMX enabled by default
Using config: /usr/lib/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@cloudera:~# /usr/lib/zookeeper/bin/zkServer.sh status
JMX enabled by default
Using config: /usr/lib/zookeeper/bin/../conf/zoo.cfg
Mode: standalone
 
There's no unit file for systemd neither startup script for SYSV neither in Cloudera's package nor Debian. So the instructions at Cloudera's documentation suggesting to run chkconfig zookeeper-server on will not work at all on Debian-based systems (even if you install chkconfig).
 
6) Install Resource Manager:
 
 
apt-get install hadoop-yarn-resourcemanager
Works fine and brings the service up.
 
7) Install HDFS namenode
 
apt-get install hadoop-hdfs-namenode
 
FAILS to bring the service UP because of several errors and stuff missing. Follow these instructions to unlock its setup.
 
Add your master node to /etc/hadoop/conf/hdfs-site.xml:
 
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://<IP-OF-YOUR-MASTER-NODE:8020</value>
</property>
 
Then run: 
 
 
hadoop namenode -format
chown hdfs:hadoop /var/lib/hadoop-hdfs/cache/
chown -R hdfs: /var/lib/hadoop-hdfs/cache/hdfs
chmod 1777 /var/lib/hadoop-hdfs/cache
service hadoop-hdfs-namenode start
 
And now we have a hadoop-namenode up and running. 

Spoiler
Please, confirm me that these permissions are fine. I've checked two other hadoop setups from Cloudera but I can't ensure it, that 1777 looks a bit too open for my taste. It can work without it but I haven't tested properly.
 
Then keep installing as explained on the official documentation, it worked for me :-)




2 REPLIES 2

Re: How to Install Cloudera on Debian 8 (probably Ubuntu as well)

Contributor

Hi,

 

Thanks for reporting that. I'm myself a big Debian user/fan (for the last 20 years) ;)

 

Your detailed steps will help others!

 

The issue with the file https://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh/cloudera.list  as already been reported internally and should be fixed very soon (couple of days).

 

JMS

Re: How to Install Cloudera on Debian 8 (probably Ubuntu as well)

New Contributor

Please try it on a cluster environment if you can, so we can ensure it works properly. My tests were done on a single-server setup.

Thanks!