Possible Accumulo-1.4.3 compatibility issues with hadoop 2.0 in CDH4 QuickStart

[#1]

 I followed the instructions at http://www.cloudera.com/content/cloudera-content/cloudera-docs/Accumulo/latest/PDF/Apache-Accumulo-I... I used the accumulo-1.4.3-cdh4.3.0-dist.tar.gz

When I ran the hellow world example to determine if the installation was correct I got the following errors.

./bin/accumulo org.apache.accumulo.examples.simple.helloworld.InsertWithOutputFormat"instance" localhost:2181"username""password" hellotable

Thread"org.apache.accumulo.examples.simple.helloworld.InsertWithOutputFormat" died nulljava.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)at java.lang.reflect.Method.invoke(Method.java:597)at org.apache.accumulo.start.Main$1.run(Main.java:89)at java.lang.Thread.run(Thread.java:662)

Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.accumulo.core.util.ContextFactory.createTaskAttemptContext(ContextFactory.java:131) at org.apache.accumulo.examples.simple.helloworld.InsertWithOutputFormat.run(InsertWithOutputFormat.java:56) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.accumulo.examples.simple.helloworld.InsertWithOutputFormat.main(InsertWithOutputFormat.java:76) ... 6 more

These problems were identified by an accumulo group as as hadoop 2.0 – accumulo 1.4.3 compatibility issues.

I really want to use the cloudera CDH4  quick start. Here is a link to my error posting on stack:http://stackoverflow.com/questions/21563143/issues-with-accumulo-1-4-3-helloworld-example-on-cdh4-3- things happen …

I would appreciate your advice!

Chris.

[#2]

Hi Chris!

It looks like you have some Accumulo jars in your path that don't work with Hadoop 2 (and thus CDH4).

First, to make sure we're on the same page, earlier I walked through the following:

1. Get the Cloudera QuickStart VM with CDH 4.4.0
2. Download the Cloudera release for Apache Accumulo 1.4.3-cdh4.3.0
3. Follow the instructions for installing above via tarball
4. Walk through the example found in docs/README.helloworld

I was able to do all of the above succesfully (after adjusting the installation classpath for using a CDH installation done with packages).

Does this matches what you are trying to do?

Presuming it does, let's step through what could be wrong with your install. Can you open a terminal on your quickstart VM and run the following:

1. $ACCUMULO_HOME/bin/accumulo version
2. $ACCUMULO_HOME/bin/accumulo classpath

Please get the output of both of these commands and paste it into a gist/pastebin somewhere. For comparison, here is the output of those commands on my QuickStart VM.

[#3]

Hey Chris!

Any luck getting the output of those two commands somewhere I can see it?

[#4]

Thnaks Sean,

Not yet ...

Accumulo runs, the examples run from the existing jars, BUT, when I try to run or modify example code.

I tried mvn -Dhadoop.profile=2.0 package on command line, eclips and netbeans builds and still get the following errors, seems it can't find org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0 :

When building"examples-simple I got the following output:========================

Some dependency artifacts are not in your local repository:

your project has dependencies that are not resolved locally. Code completion in the IDE will not include classes from these dependencies or their transitive dependencies (unless they are among the open projects).

Please download the dependencies, or install them manually, if not available remotely.

The artifacts are:

 org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0

===================================================

Build however indicated success.

I then attempted to to run the project with parameters indicated at:http://accumulo.apache.org/1.4/examples/helloworld.html

newly built  org.apache.accumulo.examples.simple.helloworld.ReadData 

I then got:========================================

----------------------------------------------------------------------

Building examples-simple 1.4.3-cdh4.3.0

------------------------------------------------------------------------

The POM for org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0 is missing, no dependency information available

------------------------------------------------------------------------

BUILD FAILURE

------------------------------------------------------------------------

Total time: 10.322s

Finished at: Wed Feb 12 21:06:25 PST 2014

Final Memory: 11M/57M

------------------------------------------------------------------------

Failed to execute goal on project examples-simple: Could not resolve dependencies for project org.apache.accumulo:examples-simple:jar:1.4.3-cdh4.3.0: Failure to find org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0 in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]

To see the full stack trace of the errors, re-run Maven with the -e switch.

Re-run Maven using the -X switch to enable full debug logging.

For more information about the errors and possible solutions, please read the following articles:

[Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

==============================================================

I was asked to try accumulo Accumulo 1.4.4-cdh4.5.0 ; Note the QuickStart VM is 4.4.0, the accumulo release you have for download is 4.5.0 is this a problem (no QuickStart).  

I appreciate your help. Putting on my snow shoes and be back in 20 minutes 🙂

Chris

[#5]

Hi Chris!

Let's try to keep these threads on a single problem. It sounds like the compatibility issue that started this thread has been addressed. Could you mark it resolved?

Your current problem sounds like it's the same one we started working on when ChunkInputFormatTest failed. Could you follow up on that question with a link to the output of running the maven commandline for building? If you use the rich text editor, there's a button for pasting in code that will help make the output more readable.

In regards to the newer release, I've had no issues using it on the Quickstart VM. However, until we figure out the root cause of your problem there's little reason to change the Accumulo version.

-Sean

[#6]

---

@crigano wrote:

I tried mvn -Dhadoop.profile=2.0 package on command line, eclips and netbeans builds and still get the following errors, seems it can't find org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0 :

When building"examples-simple I got the following output:========================

Some dependency artifacts are not in your local repository:

your project has dependencies that are not resolved locally. Code completion in the IDE will not include classes from these dependencies or their transitive dependencies (unless they are among the open projects).

Please download the dependencies, or install them manually, if not available remotely.

The artifacts are:

 org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0

===================================================

Build however indicated success.

---

This particular issue sounds like an IDE problem. Specifically, it looks like the IDE isn't reading repository settings out of the pom, so it doesn't know how to find the CDH jars. You should be able to edit the pom, get the relevant repository settings, and add them to the maven settings for hte IDE to fix part of the problem.

[#7 · Accepted solution]

Ignore the build instructions for tarball at http://www.cloudera.com/content/cloudera-content/cloudera-docs/Accumulo/latest/PDF/Apache-Accumulo-I...

Use the other instructions of go to git hub if you ar building a development environment.

[#8]

Turns out, the instructions in the PDF are outdated and no longer work. I tried to follow the install via tarball step-by-step, but got the following error:

[accumulo@localhost accumulo]$ ./bin/accumulo  init

Uncaught exception: org/apache/zookeeper/KeeperException
java.lang.NoClassDefFoundError: org/apache/zookeeper/KeeperException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.accumulo.start.classloader.AccumuloClassLoader.loadClass(AccumuloClassLoader.java:375)
    at org.apache.accumulo.start.classloader.AccumuloClassLoader.loadClass(AccumuloClassLoader.java:382)
    at org.apache.accumulo.start.Main.main(Main.java:48)
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.KeeperException
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    ... 5 more

ZOOKEEPER_HOME=/usr/lib/zookeeper/

does anyone have an updated documentation that has accumulo working on cdh4.4 VM?

thx,

[#9]

Hi Johndoe!

It's probably worth you posting a top-level question about getting things to work with the latest Quickstart VM.

FWIW, I have tested the latest install instructions along with our current release (Accumulo 1.4.4-cdh4.5.0) on the 4.4 Quickstart VM and everything worked fine.

Can you post your accumulo-site.xml somewhere that I can review it?