Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to submit remote MR jobs on CDH4.7.0 quickstart VM

Unable to submit remote MR jobs on CDH4.7.0 quickstart VM

New Contributor

(cross-posted from http://stackoverflow.com/questions/28572088/unable-to-submit-remote-mr-jobs-on-cdh4-7-0-quickstart-v...

 

I'm running a local HBase install in a VM (I'm using CDH4.7.0 available here).

 

I'm trying to submit a Hadoop job to this VM from another machine. The job that I'm submitting is:

HADOOP_USER_NAME=cloudera hadoop jar /usr/lib/hbase/hbase-0.94.15-cdh4.7.0-security.jar importtsv -Dimporttsv.separator=, -Dimporttsv.bulk.output=output -Dimporttsv.columns=HBASE_ROW_KEY,f:count wordcount /tmp/word_count.csv

The error message I'm receiving is:

15/02/17 21:48:03 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
15/02/17 21:48:03 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1075)

which tells me that this is a zookeeper configuration problem. I've ensured that both the Hbase cluster and the client have CDH4.7.0 libraries available to them. Commandline details are here.

I've copied /etc/hbase/conf, /etc/hadoop/conf and /etc/zookeeper/conf from the cluster to the client.

 

From the client, I can telnet <cluster host> 2181 so I know I'm able to connect to the VM.

 

How can I fix this zookeeper problem?

1 REPLY 1

Re: Unable to submit remote MR jobs on CDH4.7.0 quickstart VM

New Contributor

I can also do HDFS operations:

vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmpFound 4 items
-rw-r--r--   3 hdfs   supergroup          0 2015-02-17 19:21 /tmp/empty.file
drwxr-xr-x   - hdfs   supergroup          0 2014-06-01 23:15 /tmp/hadoop-yarn
drwxr-xr-x   - mapred supergroup          0 2014-06-01 23:16 /tmp/mapred
-rw-r--r--   3 hdfs   supergroup       7217 2015-02-17 19:23 /tmp/word_count.csv

vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -touchz tmp/another.empty.file
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmp
Found 5 items
-rw-r--r--   3 vagrant supergroup          0 2015-02-17 22:43 /tmp/another.empty.file
-rw-r--r--   3 hdfs    supergroup          0 2015-02-17 19:21 /tmp/empty.file
drwxr-xr-x   - hdfs    supergroup          0 2014-06-01 23:15 /tmp/hadoop-yarn
drwxr-xr-x   - mapred  supergroup          0 2014-06-01 23:16 /tmp/mapred
-rw-r--r--   3 hdfs    supergroup       7217 2015-02-17 19:23 /tmp/word_count.csv
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -get /tmp/another.empty.file
vagrant@dev-all-ubuntu-1204:/granary$ ll
total 354304
-rw-r--r--  1 vagrant vagrant         0 Feb 17 22:44 another.empty.file

I can do some operations in the hbase shell:

vagrant@dev-all-ubuntu-1204:/granary$ hbase shell
15/02/17 22:57:54 WARN conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.15-cdh4.7.0, rUnknown, Wed May 28 10:49:33 PDT 2014

hbase(main):001:0> list
TABLE                                                                                                                                 
wordcount                                                                                                                             
1 row(s) in 1.1700 seconds

hbase(main):002:0> scan 'wordcount'
ROW                                COLUMN+CELL                                                                                        
0 row(s) in 0.1180 seconds