About weiser

weiser · ‎02-17-2015

I can also do HDFS operations: vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmpFound 4 items -rw-r--r-- 3 hdfs supergroup 0 2015-02-17 19:21 /tmp/empty.file drwxr-xr-x - hdfs supergroup 0 2014-06-01 23:15 /tmp/hadoop-yarn drwxr-xr-x - mapred supergroup 0 2014-06-01 23:16 /tmp/mapred -rw-r--r-- 3 hdfs supergroup 7217 2015-02-17 19:23 /tmp/word_count.csv vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -touchz tmp/another.empty.file vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -ls /tmp Found 5 items -rw-r--r-- 3 vagrant supergroup 0 2015-02-17 22:43 /tmp/another.empty.file -rw-r--r-- 3 hdfs supergroup 0 2015-02-17 19:21 /tmp/empty.file drwxr-xr-x - hdfs supergroup 0 2014-06-01 23:15 /tmp/hadoop-yarn drwxr-xr-x - mapred supergroup 0 2014-06-01 23:16 /tmp/mapred -rw-r--r-- 3 hdfs supergroup 7217 2015-02-17 19:23 /tmp/word_count.csv vagrant@dev-all-ubuntu-1204:/granary$ hdfs dfs -get /tmp/another.empty.file vagrant@dev-all-ubuntu-1204:/granary$ ll total 354304 -rw-r--r-- 1 vagrant vagrant 0 Feb 17 22:44 another.empty.file I can do some operations in the hbase shell: vagrant@dev-all-ubuntu-1204:/granary$ hbase shell 15/02/17 22:57:54 WARN conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 0.94.15-cdh4.7.0, rUnknown, Wed May 28 10:49:33 PDT 2014 hbase(main):001:0> list TABLE wordcount 1 row(s) in 1.1700 seconds hbase(main):002:0> scan 'wordcount' ROW COLUMN+CELL 0 row(s) in 0.1180 seconds

weiser · ‎02-17-2015

(cross-posted from http://stackoverflow.com/questions/28572088/unable-to-submit-remote-mr-jobs-on-cdh4-7-0-quickstart-vm) I'm running a local HBase install in a VM (I'm using CDH4.7.0 available here). I'm trying to submit a Hadoop job to this VM from another machine. The job that I'm submitting is: HADOOP_USER_NAME=cloudera hadoop jar /usr/lib/hbase/hbase-0.94.15-cdh4.7.0-security.jar importtsv -Dimporttsv.separator=, -Dimporttsv.bulk.output=output -Dimporttsv.columns=HBASE_ROW_KEY,f:count wordcount /tmp/word_count.csv The error message I'm receiving is: 15/02/17 21:48:03 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 15/02/17 21:48:03 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1075) which tells me that this is a zookeeper configuration problem. I've ensured that both the Hbase cluster and the client have CDH4.7.0 libraries available to them. Commandline details are here. I've copied /etc/hbase/conf, /etc/hadoop/conf and /etc/zookeeper/conf from the cluster to the client. From the client, I can telnet <cluster host> 2181 so I know I'm able to connect to the VM. How can I fix this zookeeper problem?

weiser · ‎02-13-2015

Downloading the file again worked. (Though, as a small feature request, it'd be handy to have tar.gz versions of the downloads as those utilities already come with the OSes I use:))

weiser · ‎02-13-2015

The sha1 of the file I got was: vagrant@dev-all-ubuntu-1204:/granary$ sha1sum cloudera-quickstart-vm-4.7.0-0-vmware.7z c5e290f1eeb5ba588603f919e7144e6f8d41eba9 cloudera-quickstart-vm-4.7.0-0-vmware.7z Which is different from the sha1 of 626595ef7e445ecfb80280b9353340e0e049807c (from http://www.cloudera.com/content/cloudera/en/downloads/quickstart_vms/cdh-4-7-x.html) So, I'll try to download the file again.

weiser · ‎02-13-2015

Thinking that this may have been a osx issue, I tried to extract this archive in a linux VM: vagrant@dev-all-ubuntu-1204:~$ sudo apt-get install p7zip-full Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: p7zip-rar The following NEW packages will be installed: p7zip-full 0 upgraded, 1 newly installed, 0 to remove and 93 not upgraded. Need to get 1,561 kB of archives. After this operation, 3,995 kB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu/ precise/universe p7zip-full amd64 9.20.1~dfsg.1-4 [1,561 kB] Fetched 1,561 kB in 1s (934 kB/s) Selecting previously unselected package p7zip-full. (Reading database ... 81098 files and directories currently installed.) Unpacking p7zip-full (from .../p7zip-full_9.20.1~dfsg.1-4_amd64.deb) ... Processing triggers for man-db ... Setting up p7zip-full (9.20.1~dfsg.1-4) ... vagrant@dev-all-ubuntu-1204:~$ which 7za /usr/bin/7za vagrant@dev-all-ubuntu-1204:/granary$ 7za e cloudera-quickstart-vm-4.7.0-0-vmware.7z 7-Zip (A) [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) Processing archive: cloudera-quickstart-vm-4.7.0-0-vmware.7z Error: Can not open file as archive

weiser · ‎02-13-2015

When I downloaded "cloudera-quickstart-vm-4.7.0-0-vmware.7z" and opening it with The Unarchiver on OSX 10.9.5, I get an error: "There was a problem while reading the contents of the file 'cloudera-quickstart-vm-4.7.0-0-vmware.7z': The archive is incomplete" Is there any way to get a .gz version of the download?

Online	Offline
Last Visited	‎02-25-2015 12:52 PM

Date Registered	‎02-13-2015 07:55 AM
Last Visited	‎02-25-2015 12:52 PM
Total Messages Posted	6

Re: Unable to submit remote MR jobs on CDH4.7.0 qu...

Unable to submit remote MR jobs on CDH4.7.0 quicks...

Re: Issue with Quickstart VM

Re: Issue with Quickstart VM

Re: Issue with Quickstart VM

Re: Issue with Quickstart VM