Problem configuring Accumulo in CM managed cluster

[#1]

Hi,

I just wanted to install Accumulo (1.5.0 from tar.gz) on a Hadoop cluster (latest CDH4.5 parcel) managed by CM (4.8).

How do I have to set the HADOOP_HOME, ZOOKEEPER_HOME and HADOOP_CONF_DIR properties accordingly, to be able to start Accumulo in this CM-managed environment.

My current config looks like:

HADOOP_PREFIX=/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30

HADOOP_CONF_DIR="/etc/hadoop/conf"

ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/zookeeper

...but after executing "./bin/accumulo init" I get the following error:

"

root@hadoop-pg-2:~/accumulo/accumulo-1.5.0# bin/accumulo init
Uncaught exception: null
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.accumulo.start.Main.main(Main.java:41)
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.apache.commons.vfs2.impl.DefaultFileSystemManager.<init>(DefaultFileSystemManager.java:120)
at org.apache.accumulo.start.classloader.vfs.FinalCloseDefaultFileSystemManager.<init>(FinalCloseDefaultFileSystemManager.java:21)
at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.generateVfs(AccumuloVFSClassLoader.java:227)
at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.getClassLoader(AccumuloVFSClassLoader.java:201)
... 5 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:241)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

"

What's going wrong here?

I'm trying the install instructions from http://accumulo.apache.org/1.5/accumulo_user_manual.html#_installation

thanks...Gerd...

[#2]

Hi!

The problem you're running into is because the default Hadoop 2 paths (like those given in the comments of hte Apache Accumulo 1.5.0 release) aren't compatible with the layout used for libraries in CDH4.

If you take a look at this gist of an example set of configuration files for getting Accumulo 1.5.0 working on the Cloudera Qui... it should give you  a good basis for setting up an appropriate general.classpaths in your accumulo-site.xml.

One way to debug this as you go is to use the `accumulo classpath` command to see what libaries Accumulo thinks are configured. When everything is working correctly, you should see a bunch of entries with CDH version numbers.

Finally, keep in mind that the easiest way to get Accumulo up and running on CDH4 is to use our officially supported release. If you're willing to try out the latest beta version of CM there's even a parcel based installation option. Our release is based on upstream Apache Accumulo 1.4.3 (but with  Hadoop 2 support). If there's some other feature of 1.5.0 that's compelling you to go to that version, I'd love to chat about it.

-Sean

[#3]

Hi Sean,

many thanks for your reply. I tried adding the Accumulo1.4.3 parcel repo to CM, but after adding the remote parcel url (http://archive.cloudera.com/accumulo/parcels/1.4.3/) and clicking on the parcel icon I just get:

ACCUMULO 1.4.3-cdh4.3.0

Unavailable 

This is most probably due to the CDH version, I'm running CDH 4.5.0 ?!?! Is there a parcel repo available which is working in CDH4.5.0 (CM version 4.8) ?

Nevertheless, after adding

$HADOOP_PREFIX/lib/hadoop/lib/.*.jar,
$HADOOP_PREFIX/lib/hadoop/client/.*.jar,

to accumulo-site.xml I was able to proceed with executing "su hdfs ./bin/accumulo init". 

After starting accumulo (./bin/start-all.sh) as root it is even possible to login to the accumulo shell via "./bin/accumulo shell -u root".

Regrettably the web interface isn't working => "http://accumulomaster:50095/status" gives me "webpage not available" and a "telnet accumulomaster 50095" produces: "telnet: Unable to connect to remote host: Connection refused"

Any hints where I can further check what happened to the web ui ?

In the output of "accumulo classpath" command there are many lines including "/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/" on level 3, thereby I assume the config should be fine...

best regards..: Gerd :..

[#4 · Accepted solution]

FYI: after resolving some port conflicts  all problems are gone and Accumulo is up and running 

[#5]

Glad to hear everything is workign now!

The problem with the Accumulo parcels is your Cloudera Manager version. The Accumulo integration uses a new extensibility feature only available in Cloudera Manager 5's current beta release.