Member since
01-08-2014
88
Posts
15
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5914 | 10-29-2015 10:12 AM | |
6110 | 11-27-2014 11:02 AM | |
5942 | 11-03-2014 01:49 PM | |
3317 | 09-30-2014 11:26 AM | |
8781 | 09-21-2014 11:24 AM |
06-25-2014
10:43 AM
Hiya! That ticket is marked as a duplicat of HDFS-3405, which is fixed in CDH5 (5.0.0+). You can see this by looking at the release notes for HDFS in CDH 5.0.0 and seraching for HDFS-3405.
... View more
06-16-2014
06:45 AM
1 Kudo
In 1.5.x, the Accumulo scripts rely on HADOOP_PREFIX to load the hadoop native libraries. They ignore LD_LIBRARY_PATH entirely. On a parcel install you should do something like the example we ship with 1.4.4-cdh4.5.0 accumulo-env.sh: if [ -z "$HADOOP_HOME" ]
then
test -z "$HADOOP_PREFIX" && export HADOOP_PREFIX=/opt/cloudera/parcels/CDH/lib/hadoop
export HADOOP_HOME=$HADOOP_PREFIX
else
HADOOP_PREFIX="$HADOOP_HOME"
fi
test -z "$HADOOP_CLIENT_HOME" && export HADOOP_CLIENT_HOME=/opt/cloudera/parcels/CDH/lib/hadoop/client-0.20
test -z "$HADOOP_CONF_DIR" && export HADOOP_CONF_DIR="$HADOOP_PREFIX/etc/hadoop"
test -z "$ZOOKEEPER_HOME" && export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH/lib/zookeeper accumulo-site.xml: <property>
<name>general.classpaths</name>
<value>
$ACCUMULO_HOME/lib/[^.].*.jar,
$ZOOKEEPER_HOME/zookeeper[^.].*-[0-9].*.jar,
$HADOOP_CONF_DIR,
$HADOOP_CLIENT_HOME/[^.].*-[0-9].*.jar,
$HADOOP_PREFIX/lib/[^.].*.jar,
</value>
<description>Classpaths that accumulo checks for updates and class files.
When using the Security Manager, please remove the ".../target/classes/" values.
</description>
</property> In particular, note that HADOOP_PREFIX points at the lib dir for Hadoop within the CDH parcel, rather than the top level CDH dir. That last line in the classpath for HADOOP_PREFIX/lib is just to get around ACCUMULO-2301 / ACCUMULO-2786. So you could further limit it to just be the jars needed to satisfy the jetty dependency for the Monitor. If you're doing this as a new install, you're better off just running 1.6.0. It updated how native library loading was done to more closely match the expectations of Hadoop (and just general native code loading), so it goes much smoother with parcels.
... View more
03-19-2014
07:06 AM
1 Kudo
That's the problem then. The parcel for Accumulo 1.4.4-cdh4.5.0 relies on the extensibility additions in Cloudera Manager 5 beta 2. Can you update your Cloudera Manager version and try again? While Cloudera Manager 5 is in beta, it can happily install and manage any of our stable CDH releases, including the recommended version for running our latest Accumulo release: CDH 4.5.0.
... View more
03-14-2014
10:16 PM
Hi Johndoe! It's probably worth you posting a top-level question about getting things to work with the latest Quickstart VM. FWIW, I have tested the latest install instructions along with our current release (Accumulo 1.4.4-cdh4.5.0) on the 4.4 Quickstart VM and everything worked fine. Can you post your accumulo-site.xml somewhere that I can review it?
... View more
02-19-2014
11:56 AM
Your accumulo installation does not have the example configuration directories. ($ACCUMULO_HOME/conf/examples). Looking at our docs I see we tell people to move the conf directory to set up initial configs, probably on the assumption that people won't be rebuilding. I'll file an internal ticket to get that corrected. In the mean time, I'd recommend restoring the $ACCUMULO_HOME/conf directory from the dist tarball.
... View more
02-19-2014
11:36 AM
Hi Chris! Can you restore your Accumulo installation to pristine and then test? You should generally try to isolate things so that you can test one issue at a time. One way I do this is to take a snapshot of the VM after I verify any particular major change (i.e. finishing the installation instructions for Accumulo on the QuickStart VM).
... View more
02-17-2014
03:41 PM
The issue here looks like you previously did a build with the cloudera user. That build added all of the target/ directories for maven with persmissions exclusive to the cloudera user (for writing). So now when the Accumulo user attempts to build, it can't write. To fix this for now: Stop Accumulo. The rest of these instructions will delete the set of jars Accumulo needs to run. As the cloudera user run these commands to clear out the directories [cloudera@localhost ~]$ cd /usr/lib/accumulo
[cloudera@localhost accumulo]$ mvn -Dhadoop.profile=2.0 clean
[INFO] Scanning for projects...
<....skip Maven downloading the internet....>
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] accumulo .......................................... SUCCESS [0.745s]
[INFO] cloudtrace ........................................ SUCCESS [0.010s]
[INFO] accumulo-start .................................... SUCCESS [0.005s]
[INFO] accumulo-core ..................................... SUCCESS [0.040s]
[INFO] accumulo-server ................................... SUCCESS [0.035s]
[INFO] accumulo-examples ................................. SUCCESS [0.027s]
[INFO] examples-simple ................................... SUCCESS [0.014s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.345s
[INFO] Finished at: Mon Feb 17 14:51:22 PST 2014
[INFO] Final Memory: 6M/57M
[INFO] ------------------------------------------------------------------------
[cloudera@localhost accumulo]$ If that fails, you might have a mix of permissions between the cloudera and accumulo user. Run the clean command from above as root. Repeat the command line building from before as the accumulo user Start Accumulo again To fix this sustainably, you need to separate out building things from deploying them. This is especially true if you want to use a graphic IDE as the cloudera user. To do that, I'd recommend Place the code you want to work on somewhere for the cloudera user to interact with. I generally do this in a subdirectory of the home directory. [cloudera@localhost ~]$ mkdir -p projects/accumulo
[cloudera@localhost ~]$ tar -C projects/accumulo --strip=1 -xzf accumulo-1.4.3-cdh4.3.0-dist.tar.gz
[cloudera@localhost ~]$ cd projects/accumulo/
[cloudera@localhost accumulo]$ ls -a
. .. bin CHANGES cloudera conf contrib docs lib LICENSE logs NOTICE pom.xml README src test walogs
[cloudera@localhost accumulo]$ Out of habit, I would track whatever changes happen locally [cloudera@localhost accumulo]$ git config --global user.name "Sean Busbey"
[cloudera@localhost accumulo]$ git config --global user.email "busbey@cloudera.com"
[cloudera@localhost accumulo]$ git init
Initialized empty Git repository in /home/cloudera/projects/accumulo/.git/
[cloudera@localhost accumulo]$ git commit --allow-empty -m "Initial commit"
[master (root-commit) 6943bc0] Initial commit
[cloudera@localhost accumulo]$ git add *
[cloudera@localhost accumulo]$ git commit -m "Cloudera release 1.4.3-cdh4.3.0"
[master b4a9911] Cloudera release 1.4.3-cdh4.3.0
1551 files changed, 338162 insertions(+), 0 deletions(-)
<.... skip 1551 file paths ....>
[cloudera@localhost accumulo]$ git status
# On branch master
nothing to commit (working directory clean)
[cloudera@localhost accumulo]$ git log
commit b4a99119da3de3f9445b793ebd338dcbb310ace3
Author: Sean Busbey <busbey@cloudera.com>
Date: Mon Feb 17 15:05:50 2014 -0800
Cloudera release 1.4.3-cdh4.3.0
commit 6943bc00904b42c5ee6aec472e963f0188433267
Author: Sean Busbey <busbey@cloudera.com>
Date: Mon Feb 17 15:05:30 2014 -0800
Initial commit
[cloudera@localhost accumulo]$ Build however you're going to build (IDE or commandline). As an example, I would build via commandline maven. [cloudera@localhost accumulo]$ mvn -Dhadoop.profile=2.0 package
[INFO] Scanning for projects...
[WARNING]
<.... Skip warnings about plugin versions ....>
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] accumulo
[INFO] cloudtrace
[INFO] accumulo-start
[INFO] accumulo-core
[INFO] accumulo-server
[INFO] accumulo-examples
[INFO] examples-simple
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building accumulo 1.4.3-cdh4.3.0
[INFO] ------------------------------------------------------------------------
<.... skip maven downloading the internet ....>
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] accumulo .......................................... SUCCESS [3:11.272s]
[INFO] cloudtrace ........................................ SUCCESS [1:58.342s]
[INFO] accumulo-start .................................... SUCCESS [4:35.473s]
[INFO] accumulo-core ..................................... SUCCESS [59.175s]
[INFO] accumulo-server ................................... SUCCESS [1:53.334s]
[INFO] accumulo-examples ................................. SUCCESS [0.009s]
[INFO] examples-simple ................................... SUCCESS [7.238s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:46.212s
[INFO] Finished at: Mon Feb 17 15:21:16 PST 2014
[INFO] Final Memory: 80M/192M
[INFO] ------------------------------------------------------------------------
[cloudera@localhost accumulo]$ When you need to deploy the jars into your accumulo instance: replace the Accumulo deployment jars with those from the working directory and restart Accumulo (all with the accumulo user). [cloudera@localhost accumulo]$ sudo -u accumulo rm -f /usr/lib/accumulo/lib/*.jar
[cloudera@localhost accumulo]$ sudo -u accumulo cp lib/*.jar /usr/lib/accumulo/lib/
[cloudera@localhost accumulo]$ sudo su - accumulo
[accumulo@localhost ~]$ $ACCUMULO_HOME/bin/stop-all.sh
^CInvalid password or unable to connect to the master
Press Ctrl-C to cancel now, or force shutdown in 15 seconds
Utilities and unresponsive servers will be shut down in 5 seconds
stopping monitor on localhost.localdomain
stopping tracer on localhost.localdomain
stopping monitor on localhost.localdomain
stopping unresponsive tablet servers (if any) ...
stopping unresponsive tablet servers hard (if any) ...
Cleaning tablet server and logger entries from zookeeper
Cleaning all server entries in zookeeper
[accumulo@localhost ~]$ $ACCUMULO_HOME/bin/start-all.sh
Starting tablet servers and loggers .... done
Starting tablet server on localhost.localdomain
Starting logger on localhost.localdomain
2014-02-17 15:34:41,763 [server.Accumulo] INFO : Attempting to talk to zookeeper
2014-02-17 15:34:42,136 [server.Accumulo] INFO : Zookeeper connected and initialized, attemping to talk to HDFS
2014-02-17 15:34:47,210 [server.Accumulo] INFO : Connected to HDFS
Starting master on localhost.localdomain
Starting garbage collector on localhost.localdomain
Starting monitor on localhost.localdomain
Starting tracer on localhost.localdomain
[accumulo@localhost ~]$ exit
logout
[cloudera@localhost accumulo]$ Note that this process is only needed because you're altering the jars that make up Accumulo itself. Under normal operations, you should not be altering this source nor copying jars into Accumulo's installation. Any jars you need, either for client side applications or for a custom iterator, should be deployed in their own location and referenced as appropriate from your client or server classpaths.
... View more
02-17-2014
03:21 PM
Yes. Those entries are so you can test changes to the Accumulo code by recompiling and restarting Accumulo (i.e. skipping the step of building jars and deploying them). I generally recommend never including those paths in Accumulo's classpath because They should never be present in a production deployment and it's a bad idea to intentionally introduce differences between dev and production (in this case being packaged in jars) The main code base doesn't build unless you use the package goal anyways, so you'll already have jars ready to go If you are building directly within your Accumulo installation (which you should not do), the default package goal will give you your changes loaded upon restart just as easily, because the package goal will have put the repackaged Accumulo jars into the lib directory.
... View more
02-14-2014
01:06 PM
Hi Chris! Yep, it's almost certainly a local filesystem permissions issue. Please run the commands in my previous post. They will show the permissions at likely problem points. -Sean
... View more