Reply
Explorer
Posts: 8
Registered: ‎03-06-2018

install hadoop client on unmanaged host

[ Edited ]

Hello,

 

I am trying to install the Hadoop client components on a host that is not managed by Cloudera Manager. After doing some digging, some people suggested that simply installing the hadoop-client, and adding the site configuration files should do the trick.

 

But I can't find hadoop-client!

 

Here is yum repo:

 

 

[cloudera-manager]
name = Cloudera Manager, Version 5.12.1
baseurl = https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.12.1/
gpgkey = https://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera
gpgcheck = 1

 

 

And the output of yum:

 

 

$ sudo yum install hadoop-client
Loaded plugins: fastestmirror
...
cloudera-manager                               |  951 B  00:00:00     
...
...
cloudera-manager/primary                       | 4.3 kB  00:00:00     
...
...
cloudera-manager                               7/7
No package hadoop-client available.
Error: Nothing to do

Your help is appreciated.

Highlighted
Posts: 582
Topics: 1
Kudos: 138
Solutions: 74
Registered: ‎04-22-2014

Re: install hadoop client on unmanaged host

Hi @ramin,

 

Here are some general instructions I found internally. Note: You can change the path to match the OS release and CDH version of the client you need.

 

  1. On the external host download the CDH repo file to the /etc/yum.repos.d/ directory:
    curl -O https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/cloudera-cdh5.repo
  2. Edit the base URL in the cloudera-cdh5.repo file to install the CDH version (otherwise it will install the latest). For example, to install the 5.7.1 hadoop-client, update the baseurl to:
    baseurl=https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.7.1/
  3. Install the hadoop-client rpm:
    $ yum clean all
    $ yum install hadoop-client

       (See http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.7.1/RPMS/x86_64/)

  1. In Cloudera Manager navigate to, HDFS -> "Actions" drop down -> "Download Client Configuration" (this will download a zip file called hdfs-clientconfig.zip).
  2. Move the zip file over to the external host and unzip it.
  3. Copy all the unzipped configuration files to /etc/hadoop/conf. Example:
    $ cp *  /etc/hadoop/conf
  4. Run hadoop commands. Example:
    $ sudo -u hdfs hadoop fs -ls

Note: You can also download the RPM file and install locally if desired.

 

Announcements