Member since
02-13-2015
6
Posts
0
Kudos Received
0
Solutions
02-17-2015
03:01 PM
I can also do HDFS operations: vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmpFound 4 items
-rw-r--r-- 3 hdfs supergroup 0 2015-02-17 19:21 /tmp/empty.file
drwxr-xr-x - hdfs supergroup 0 2014-06-01 23:15 /tmp/hadoop-yarn
drwxr-xr-x - mapred supergroup 0 2014-06-01 23:16 /tmp/mapred
-rw-r--r-- 3 hdfs supergroup 7217 2015-02-17 19:23 /tmp/word_count.csv
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -touchz tmp/another.empty.file
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmp
Found 5 items
-rw-r--r-- 3 vagrant supergroup 0 2015-02-17 22:43 /tmp/another.empty.file
-rw-r--r-- 3 hdfs supergroup 0 2015-02-17 19:21 /tmp/empty.file
drwxr-xr-x - hdfs supergroup 0 2014-06-01 23:15 /tmp/hadoop-yarn
drwxr-xr-x - mapred supergroup 0 2014-06-01 23:16 /tmp/mapred
-rw-r--r-- 3 hdfs supergroup 7217 2015-02-17 19:23 /tmp/word_count.csv
vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -get /tmp/another.empty.file
vagrant@dev-all-ubuntu-1204:/granary$ ll
total 354304
-rw-r--r-- 1 vagrant vagrant 0 Feb 17 22:44 another.empty.file I can do some operations in the hbase shell: vagrant@dev-all-ubuntu-1204:/granary$ hbase shell
15/02/17 22:57:54 WARN conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.15-cdh4.7.0, rUnknown, Wed May 28 10:49:33 PDT 2014
hbase(main):001:0> list
TABLE
wordcount
1 row(s) in 1.1700 seconds
hbase(main):002:0> scan 'wordcount'
ROW COLUMN+CELL
0 row(s) in 0.1180 seconds
... View more
02-17-2015
02:16 PM
(cross-posted from http://stackoverflow.com/questions/28572088/unable-to-submit-remote-mr-jobs-on-cdh4-7-0-quickstart-vm) I'm running a local HBase install in a VM (I'm using CDH4.7.0 available here). I'm trying to submit a Hadoop job to this VM from another machine. The job that I'm submitting is: HADOOP_USER_NAME=cloudera hadoop jar /usr/lib/hbase/hbase-0.94.15-cdh4.7.0-security.jar importtsv -Dimporttsv.separator=, -Dimporttsv.bulk.output=output -Dimporttsv.columns=HBASE_ROW_KEY,f:count wordcount /tmp/word_count.csv The error message I'm receiving is: 15/02/17 21:48:03 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
15/02/17 21:48:03 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1075) which tells me that this is a zookeeper configuration problem. I've ensured that both the Hbase cluster and the client have CDH4.7.0 libraries available to them. Commandline details are here. I've copied /etc/hbase/conf, /etc/hadoop/conf and /etc/zookeeper/conf from the cluster to the client. From the client, I can telnet <cluster host> 2181 so I know I'm able to connect to the VM. How can I fix this zookeeper problem?
... View more
02-13-2015
09:13 AM
Downloading the file again worked. (Though, as a small feature request, it'd be handy to have tar.gz versions of the downloads as those utilities already come with the OSes I use:))
... View more
02-13-2015
08:46 AM
The sha1 of the file I got was: vagrant@dev-all-ubuntu-1204:/granary$ sha1sum cloudera-quickstart-vm-4.7.0-0-vmware.7z c5e290f1eeb5ba588603f919e7144e6f8d41eba9 cloudera-quickstart-vm-4.7.0-0-vmware.7z Which is different from the sha1 of 626595ef7e445ecfb80280b9353340e0e049807c (from http://www.cloudera.com/content/cloudera/en/downloads/quickstart_vms/cdh-4-7-x.html) So, I'll try to download the file again.
... View more
02-13-2015
08:04 AM
Thinking that this may have been a osx issue, I tried to extract this archive in a linux VM: vagrant@dev-all-ubuntu-1204:~$ sudo apt-get install p7zip-full Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: p7zip-rar The following NEW packages will be installed: p7zip-full 0 upgraded, 1 newly installed, 0 to remove and 93 not upgraded. Need to get 1,561 kB of archives. After this operation, 3,995 kB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu/ precise/universe p7zip-full amd64 9.20.1~dfsg.1-4 [1,561 kB] Fetched 1,561 kB in 1s (934 kB/s) Selecting previously unselected package p7zip-full. (Reading database ... 81098 files and directories currently installed.) Unpacking p7zip-full (from .../p7zip-full_9.20.1~dfsg.1-4_amd64.deb) ... Processing triggers for man-db ... Setting up p7zip-full (9.20.1~dfsg.1-4) ... vagrant@dev-all-ubuntu-1204:~$ which 7za /usr/bin/7za vagrant@dev-all-ubuntu-1204:/granary$ 7za e cloudera-quickstart-vm-4.7.0-0-vmware.7z 7-Zip (A) [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) Processing archive: cloudera-quickstart-vm-4.7.0-0-vmware.7z Error: Can not open file as archive
... View more
02-13-2015
07:58 AM
When I downloaded "cloudera-quickstart-vm-4.7.0-0-vmware.7z" and opening it with The Unarchiver on OSX 10.9.5, I get an error: "There was a problem while reading the contents of the file 'cloudera-quickstart-vm-4.7.0-0-vmware.7z': The archive is incomplete" Is there any way to get a .gz version of the download?
... View more