Reply
Expert Contributor
Posts: 126
Registered: ‎11-01-2013

How to access multiple hadoop clusters from one remote machine

I have 3 seperate clusters, I wanna setup one client machince which can talk to all the three clusters, Anyone have any idea on how to acheave this ?

 

Any help greatly appreciated.

 

 

Em Jay
Expert Contributor
Posts: 126
Registered: ‎11-01-2013

Re: How to access multiple hadoop clusters from one remote machine

[ Edited ]

By the way, I don't want to use CM API..

Em Jay
Cloudera Employee
Posts: 578
Registered: ‎01-20-2014

Re: How to access multiple hadoop clusters from one remote machine

Why not download client configuration from all three clusters into three
directories. Then switch /etc/hadoop/conf to point to the one you want to
talk to.

Regards,
Gautam Gopalakrishnan
Expert Contributor
Posts: 126
Registered: ‎11-01-2013

Re: How to access multiple hadoop clusters from one remote machine

[ Edited ]

Correct. I can do that. I want to know are there any other ways to talk to multiple clusters because like you said I do not have enough permissions on the server to create client configs and link to /etc/hadoop/conf

Em Jay
Cloudera Employee
Posts: 578
Registered: ‎01-20-2014

Re: How to access multiple hadoop clusters from one remote machine

Why not use HADOOP_CONF_DIR instead?

http://wiki.apache.org/hadoop/HowToConfigure

Regards,
Gautam Gopalakrishnan
Expert Contributor
Posts: 126
Registered: ‎11-01-2013

Re: How to access multiple hadoop clusters from one remote machine

Can you elaborate the solution ?
Place three cluster configs in a location and use HADOOP_CONF_DIR variable to pick certain values from the configs and communicate the cluster ? Is that what you mean ?
Em Jay
Cloudera Employee
Posts: 578
Registered: ‎01-20-2014

Re: How to access multiple hadoop clusters from one remote machine

Basically same as earlier. Except switching symlink under /etc, you point
to the directory using HADOOP_CONF_DIR on the shell. Please try and let me
know if it works

Regards,
Gautam Gopalakrishnan
Announcements