Member since
07-31-2013
10
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
23347 | 03-01-2014 08:03 AM | |
18197 | 08-06-2013 05:36 PM |
03-01-2014
12:58 PM
For anybody who might be following this, I referenced all the libraries referenced in the video and for HBase , added the 2 HBase refs to the reference list (hbase.jar and hbase-0.92.1-cdh4.1.1-security.jar) Then it worked in Eclipse. -- jp
... View more
03-01-2014
08:03 AM
Was able to run this task: [training@localhost java]$ javac -cp `hbase classpath` MakeTable.java
[training@localhost java]$ java -cp `hbase classpath` MakeTable Thanks for your help. jp
... View more
02-28-2014
08:10 PM
Thanks. I used the excellent video and blog (and my Cloudera Academic Training notes) to run Hadoop MR scenarios as jar files and through Eclipse. Very happy. I haven't been successful in running the hbase java program I listed earlier, or the MakeTable example included in the hbase directory. Configuration issues? (I'm using the out-of-the-box configuration.) [training@localhost java]$ javac -classpath `hbase classpath` MakeTable.java
[training@localhost java]$ jar cvf MakeTable.jar MakeTable.class
[training@localhost java]$ hadoop jar MakeTable.jar MakeTable Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration at MakeTable.main(MakeTable.java:24) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 6 more jp
... View more
02-27-2014
06:17 PM
I'm trying to run it through the IDE (Eclipse) - just running the program in Debug mode. In my search for answers, I'm only seeing questions from people running from the command line. Can I test from the Eclipse IDE, or will my testing have to be from jar files at the command line?
... View more
02-26-2014
08:04 PM
I'm able to run simple 'create' and 'put' to add data to rows (I'm using HBase - the Definitive Guide) but it's failing when I try to run the Java program shown below: import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
public class PutExample {
public static void main(String[] args) throws IOException {
Configuration conf = HBaseConfiguration.create();
HTable table = new HTable(conf, "jptesttable");
Put put = new Put(Bytes.toBytes("row1"));
put.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
put.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));
table.put(put);
table.close();
}
} I'm pretty sure I have all the libraries I need (plus some I don't need): hbase.jar commons-logging-1.1.1.jar log4j-1.2.17.jar zookeeper.jar commons-lang-2.5.jar commons-configuration-1.6.jar slf4j-api-1.6.1.jar slf4j-log4j12-1.6.1.jar hadoop.core.jar core.3.1.1.jar hadoop.common-2.0.0-cdh4.1.1.jar hbase-0.92.1-cdh4.1.1-security.jar and usr/lib/hbase/conf in the path I haven't changed the hbase.site.xml file: <configuration>
<!-- Changing the default port for REST since it conflicts with yarn nodemanager -->
<property>
<name>hbase.rest.port</name>
<value>8070</value>
<description>The port for the HBase REST server.</description>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:8020/hbase</value>
</property>
</configuration> => Is there anything I can do to get this to work? Here's the error log, Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Maps at org.apache.hadoop.metrics2.lib.MetricsRegistry.<init>(MetricsRegistry.java:42) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:87) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:133) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36) at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:97) at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:190) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37) at org.apache.hadoop.hbase.security.User.call(User.java:586) at org.apache.hadoop.hbase.security.User.callStatic(User.java:576) at org.apache.hadoop.hbase.security.User.access$400(User.java:50) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.<init>(User.java:393) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.<init>(User.java:388) at org.apache.hadoop.hbase.security.User.getCurrent(User.java:139) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionKey.<init>(HConnectionManager.java:412) at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:182) at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:196) at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:173) at PutExample.main(PutExample.java:17) Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Maps at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 23 more thanks, jp
... View more
Labels:
08-06-2013
05:36 PM
1 Kudo
It took me a while to figure out. I just got it a minute ago. I was running scripts that I developed in Windows (where end-of-line = cr+lf). I needed to strip out the "cr" so the python interpreter in Linux wouldn't be looking for /usr/bin/env python/r, but /usr/bin/env python. Now I can move on. jp
... View more
08-01-2013
09:30 PM
thanks - I did this through the properties screen of the file browser, but I tried it again with the command you supplied. still no luck - the process completes, but outputs nothing, even with a plain vanilla reducer (echoing the mapper output). jp
... View more
08-01-2013
09:22 PM
I renamed my mapper and reducer to jpm.py and jpr.py to make sure my spelling is right. The reducer part of the "cat" doesn't work unless it's preceeded by "python". Then it completes successfully. In hadoop map-reduce, from the command line, I've gotten the process to complete, but it yields no results. I reduced the reducer functionality to just pass on what comes from the mapper. It completes, but doesn't yield any results in the output (file size = 0). I removed the reducer completely and I get what I expect from the mapper. I'd like to progress to the gui's and get a taste of pig and hive in cloudera by the end of the month. I think I'm going to try all over again with a fresh vm.
... View more
08-01-2013
01:43 PM
Thanks. I rebooted, reconstructed new files and again tried both #!/usr/bin/env python and #!/usr/bin/python and changed permissions to include -x . I'm making it through the file, mymapper, and sort, but I'm getting "no such file or directory" when I pipe it to ./myreducer.py But when I explicitly add "python" as the executable it works. I'm guessing this is some obvious newbie issue (new to linux) but I should have this in the bag by now. jp
... View more
07-31-2013
09:07 PM
I'm trying to use my local installation of Cloudera Quickstart VM to do a small mapreduce job in Python. My test script works when I explicitly add python to the script: # cat inputfile.txt | python mymapper.py | sort | python myreducer.py I need to add python to the path in the vm. What's the best way to do this so it finds python from the command line and in Hadoop? I haven't been successful trying to find and modify the right files in the Cloudera VM. (I was able to run this on AWS. I tried from the hadoop command line also: hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.3.0.jar \ -input inputfile.txt \ -output output010 \ -mapper mymapper.py \ -file mymapper.py \ -combiner myreducer.py \ -reducer myreducer.py \ -file myreducer.py ... and it fails) Any help to get the right would be appreciated. thanks, jp
... View more
Labels:
- Labels:
-
Apache Hadoop
-
MapReduce
-
Quickstart VM