I would like to be able to perform hdfs commands from a computer that is NOT actually part of the cloudera cluster.
For example, performing simple put/get operations or
hdfs dfs -ls /my/dir
I have installed the correct binaries, I think. I found from CM that I was using CDH4.7.1. So, I installed (sudo apt-get install hadoop-client) the binaries from here.
If I run:
hdfs dfs -ls /
Error: JAVA_HOME is not set and could not be found.
I feel that this might just be the beginning of a long tinkering and configuring process and I unfortunately know nothing about java. I do, however, know the IPs of the namenodes on my cluster and have access to all admin rights from beginning to end.
Can someone help me get things configured?
P.S. In case it wasn't clear. I can perform all desired functionality on nodes that are part of the cluster. I just want to do something similar from my development environment.
It is part of the magic that happens when you install a gateway roles on a nodes using CM. It installs the binaries, download the client configs, sets env vars in such a way that no users on that nodes need to do it. It just works. I have not dug into it to determine how. I expected that it is wrapped up in scripts somewhere that get called when using the clients.
Without it, you will need to do it all yourself. And JAVA_HOME isn't the only one, HADOOP_HOME, HADOOP_CONF, etc.
The Hadoop docs should have some information on it.
Or... you can add the machine in question to CM, add the gateway role, and then remove it from the cluster. You shouldn't need to uninstall the gateway role to remove it from the cluster. No license is required for the gateway roles either.
Thanks a bunch! That's kind of what I feared. Glad I didn't go down too much of a rabbit hole yet.
For what it's worth, the process is really easy to get `impala-shell` working. I just added the Cloudera repository, installed with apt-get and then put the ip address of an impala node into the commands.
I was hoping to get something similar for hdfs. Oh well. I'll look into what you said.