Unable to submit remote MR jobs on CDH4.7.0 quickstart VM

[weiser]

(cross-posted from http://stackoverflow.com/questions/28572088/unable-to-submit-remote-mr-jobs-on-cdh4-7-0-quickstart-v...

I'm running a local HBase install in a VM (I'm using CDH4.7.0 available here).

I'm trying to submit a Hadoop job to this VM from another machine. The job that I'm submitting is:

HADOOP_USER_NAME=cloudera hadoop jar /usr/lib/hbase/hbase-0.94.15-cdh4.7.0-security.jar importtsv -Dimporttsv.separator=, -Dimporttsv.bulk.output=output -Dimporttsv.columns=HBASE_ROW_KEY,f:count wordcount /tmp/word_count.csv

The error message I'm receiving is:

15/02/17 21:48:03 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
15/02/17 21:48:03 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1075)

which tells me that this is a zookeeper configuration problem. I've ensured that both the Hbase cluster and the client have CDH4.7.0 libraries available to them. Commandline details are here.

I've copied /etc/hbase/conf, /etc/hadoop/conf and /etc/zookeeper/conf from the cluster to the client.

From the client, I can telnet <cluster host> 2181 so I know I'm able to connect to the VM.

How can I fix this zookeeper problem?

[weiser]

I can also do HDFS operations:

vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmpFound 4 items
-rw-r--r--   3 hdfs   supergroup          0 2015-02-17 19:21 /tmp/empty.file
drwxr-xr-x   - hdfs   supergroup          0 2014-06-01 23:15 /tmp/hadoop-yarn
drwxr-xr-x   - mapred supergroup          0 2014-06-01 23:16 /tmp/mapred
-rw-r--r--   3 hdfs   supergroup       7217 2015-02-17 19:23 /tmp/word_count.csv

vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -touchz tmp/another.empty.file
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmp
Found 5 items
-rw-r--r--   3 vagrant supergroup          0 2015-02-17 22:43 /tmp/another.empty.file
-rw-r--r--   3 hdfs    supergroup          0 2015-02-17 19:21 /tmp/empty.file
drwxr-xr-x   - hdfs    supergroup          0 2014-06-01 23:15 /tmp/hadoop-yarn
drwxr-xr-x   - mapred  supergroup          0 2014-06-01 23:16 /tmp/mapred
-rw-r--r--   3 hdfs    supergroup       7217 2015-02-17 19:23 /tmp/word_count.csv
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -get /tmp/another.empty.file
vagrant@dev-all-ubuntu-1204:/granary$ ll
total 354304
-rw-r--r--  1 vagrant vagrant         0 Feb 17 22:44 another.empty.file

I can do some operations in the hbase shell:

vagrant@dev-all-ubuntu-1204:/granary$ hbase shell
15/02/17 22:57:54 WARN conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.15-cdh4.7.0, rUnknown, Wed May 28 10:49:33 PDT 2014

hbase(main):001:0> list
TABLE                                                                                                                                 
wordcount                                                                                                                             
1 row(s) in 1.1700 seconds

hbase(main):002:0> scan 'wordcount'
ROW                                COLUMN+CELL                                                                                        
0 row(s) in 0.1180 seconds