Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest

[#1]

Greetings to all esteem colleague,

Following indicate a build failure for accumulo-examples indicating  the failure of Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.

I would very much appreciate advice on actions to take in detail that would resolve this issue.

The environment is accumulo-1.4.3, CDH4.3 QuickStart using netbeans to import the maven project  accumulo-examples after installing accumulo from the accumulo-1.4.3cdh4.3.tar.gz.

The VM is available upon request.

Cloudera has not responded in there news groups.

In advance thanks for your assistance.

The results of executing the build with dependencies option are listed below:

cd /usr/lib/accumulo/src/examples/simple; JAVA_HOME=/usr/java/jdk1.6.0_32 /home/cloudera/netbeans-7.4/java/maven/bin/mvn install

Scanning for projects...

Some problems were encountered while building the effective model for org.apache.accumulo:examples-simple:jar:1.4.3-cdh4.3.0

'build.plugins.plugin.version' for org.apache.maven.plugins:maven-clean-plugin is missing. @ org.apache.accumulo:accumulo:1.4.3-cdh4.3.0, /usr/lib/accumulo/pom.xml, line 81, column 15

'build.plugins.plugin.version' for org.apache.maven.plugins:maven-resources-plugin is missing. @ org.apache.accumulo:accumulo:1.4.3-cdh4.3.0, /usr/lib/accumulo/pom.xml, line 152, column 15

It is highly recommended to fix these problems because they threaten the stability of your build.

For this reason, future Maven versions might no longer support building such malformed projects.

------------------------------------------------------------------------

Building examples-simple 1.4.3-cdh4.3.0

------------------------------------------------------------------------

--- maven-enforcer-plugin:1.0:enforce (enforce-mvn) @ examples-simple ---

--- maven-resources-plugin:2.5:resources (default-resources) @ examples-simple ---

[debug] execute contextualize

Using 'UTF-8' encoding to copy filtered resources.

skip non existing resourceDirectory /usr/lib/accumulo/src/examples/simple/src/main/resources

--- maven-dependency-plugin:2.1:copy-dependencies (copy-dependencies) @ examples-simple ---

log4j-1.2.16.jar already exists in destination.

--- maven-compiler-plugin:2.3.2:compile (default-compile) @ examples-simple ---

Compiling 40 source files to /usr/lib/accumulo/src/examples/simple/target/classes

--- maven-resources-plugin:2.5:testResources (default-testResources) @ examples-simple ---

[debug] execute contextualize

Using 'UTF-8' encoding to copy filtered resources.

skip non existing resourceDirectory /usr/lib/accumulo/src/examples/simple/src/test/resources

--- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ examples-simple ---

Compiling 5 source files to /usr/lib/accumulo/src/examples/simple/target/test-classes

--- maven-surefire-plugin:2.9:test (default-test) @ examples-simple ---

Surefire report directory: /usr/lib/accumulo/src/examples/simple/target/surefire-reports

-------------------------------------------------------

 T E S T S

-------------------------------------------------------

Running org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest

Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 1.89 sec <<< FAILURE!

Running org.apache.accumulo.examples.simple.filedata.ChunkCombinerTest

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec

Running org.apache.accumulo.examples.simple.filedata.ChunkInputStreamTest

Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.175 sec

Running org.apache.accumulo.examples.simple.filedata.KeyUtilTest

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec

Running org.apache.accumulo.examples.simple.dirlist.CountTest

Max depth              : 3

Time to find max depth : 2 ms

Time to compute counts : 7 ms

Entries scanned        : 30 

Counts inserted        : 4 

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec

Results :

Tests in error: 

  test(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest): Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected

  testErrorOnNextWithoutClose(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest): Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected

  testInfoWithoutChunks(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest): Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected

Tests run: 15, Failures: 0, Errors: 3, Skipped: 0

------------------------------------------------------------------------

BUILD FAILURE

------------------------------------------------------------------------

Total time: 17.620s

Finished at: Tue Feb 11 14:34:18 PST 2014

Final Memory: 20M/57M

------------------------------------------------------------------------

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.9:test (default-test) on project examples-simple: There are test failures.

Please refer to /usr/lib/accumulo/src/examples/simple/target/surefire-reports for the individual test results.

-> [Help 1]

To see the full stack trace of the errors, re-run Maven with the -e switch.

Re-run Maven using the -X switch to enable full debug logging.

For more information about the errors and possible solutions, please read the following articles:

[Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

-> [Help 1]

==============================================================================

Following is: The results of the failed report in  "/usr/lib/accumulo/src/examples/simple/target/surefire-reports"

-------------------------------------------------------------------------------

Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest

-------------------------------------------------------------------------------

Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 2.297 sec <<< FAILURE!

test(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest)  Time elapsed: 1.542 sec  <<< ERROR!

java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected

        at org.apache.accumulo.core.client.mapreduce.InputFormatBase$RecordReaderBase.initialize(InputFormatBase.java:1108)

        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormat$1.initialize(ChunkInputFormat.java:47)

        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.test(ChunkInputFormatTest.java:95)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at junit.framework.TestCase.runTest(TestCase.java:168)

        at junit.framework.TestCase.runBare(TestCase.java:134)

        at junit.framework.TestResult$1.protect(TestResult.java:110)

        at junit.framework.TestResult.runProtected(TestResult.java:128)

        at junit.framework.TestResult.run(TestResult.java:113)

        at junit.framework.TestCase.run(TestCase.java:124)

        at junit.framework.TestSuite.runTest(TestSuite.java:232)

        at junit.framework.TestSuite.run(TestSuite.java:227)

        at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)

        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)

        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)

        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)

        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)

        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)

        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)

        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)

testErrorOnNextWithoutClose(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest)  Time elapsed: 0.084 sec  <<< ERROR!

java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected

        at org.apache.accumulo.core.client.mapreduce.InputFormatBase$RecordReaderBase.initialize(InputFormatBase.java:1108)

        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormat$1.initialize(ChunkInputFormat.java:47)

        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.testErrorOnNextWithoutClose(ChunkInputFormatTest.java:146)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at junit.framework.TestCase.runTest(TestCase.java:168)

        at junit.framework.TestCase.runBare(TestCase.java:134)

        at junit.framework.TestResult$1.protect(TestResult.java:110)

        at junit.framework.TestResult.runProtected(TestResult.java:128)

        at junit.framework.TestResult.run(TestResult.java:113)

        at junit.framework.TestCase.run(TestCase.java:124)

        at junit.framework.TestSuite.runTest(TestSuite.java:232)

[#2]

Hi Chris!

This particular problem is caused by attempting to build Accumulo in Hadoop 1 mode against Hadoop 2 (specifically CDH4).

In the initial Cloudera integrated release of the Accumulo 1.4.x line, you still had to specify the Hadoop 2 profile via a Java system property.

Command line maven would look like this:

$> mvn -Dhadoop.profile=2.0 package

 I presume there is some way to specify this in the build options for netbeans, but I am not familiar with that particular IDE.

Alternatively, you could install our latest release, Accumulo 1.4.4-cdh4.5.0 (available from archive), which defaults to building against the Hadoop 2 profile.

It might also help to know why you're rebuilding the examples. From your other outstanding question, my guess is that you're using it as a starting point to learn about general development?

[#3]

Exactly!

I am new to netbeans as well and am not a Maven guy.

I'll see if I can use mvn -Dhadoop.profile=2.0 package.

I will be starting on my maven book in 20 minutes (sort of a machine
learning guy)

I was using the examples to be modified during training.

I will rebuild using accumulo Accumulo
1.4.4-cdh4.5.0
if
not.

Can I still use the current QuickStart VM?

appreciate your assistance Joey,

Chris

Ps. First response from Cloudera I have gotten with 4 posts

[#4]

FYI, I have moved this thread to the Accumulo board...

[#5]

@crigano I'm dropping your previous message into this thread, as well as @busbey 's since they both seem to have gotten derailed into an offline email thread.  and FWIW, there is no Joey on this thread, not sure if there is some threads getting crossed here or what.

From busbey:

Hi Chris!

If you're not already familiar with Netbeans, I'd recommend sticking with the development tools you're comfortable with.

You should check your notification settings; I know I responded to at least one of your other and I believe someone else responded to another.

-Sean

from crigano:

put the  in the IDE: mvn -Dhadoop.profile=2.0 package

Now I get completion in the IDE will not include classes from these dependencies or their transitive dependencies (unless they are among the open projects).

Please download the dependencies, or install them manually, if not available remotely.

The artifacts are:

 org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0

What do you recommend Joey,

thanks,

Chris

[#6]

Hi Chris!

That definitely sounds like a problem with getting Netbeans to play nicely with Maven's dependency providing.

To avoid the drag of these kinds of tooling issues, I generally recommend people stick with whatever workflow they're used to for Java development.

Is there another IDE you are more familiar with?

-Sean

[#7]

My Bad,

I set up notifications to go to chris.p.rigano@gmail.com.

I am just trying to get a version of the CDH QuickStart with accumulo to
run for classes I am teaching and to do work outside the IC doors.

I tried both eclipse I have been using for a number of years and got the
right setting in netbeans.

Still not working as indicated in literature for examples.

This is not an ide issue but an environment problem.

I would be glad to send you my VM built IAW cloudera guidance.

It may make things easier.

Give me access to a fast FTP.

I used CDH at Meade for a while. I am a cloudera fan.

Best regards,

Chris

[#8]

Let's step back and confirm that your VM can properly see the necessary Maven repositories.

I'm using a QuickStart VM with CDH 4.4.0 and I have the Accumulo 1.4.3-cdh4.3.0 release installed from tarballs in /usr/lib/accumulo.

If I open up a terminal and navigate to the install directory, I can successfully rebuild everything including the examples.

[cloudera@localhost ~]$ sudo su - accumulo
[accumulo@localhost ~]$ cd /usr/lib/accumulo
[accumulo@localhost accumulo]$ mvn -Dhadoop.profile=2.0 package
<...snip maven downloading dependencies from the internet...>
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.accumulo.examples.simple.dirlist.CountTest
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Max depth              : 3
Time to find max depth : 7 ms
Time to compute counts : 7 ms
Entries scanned        : 30 
Counts inserted        : 4 
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.302 sec
Running org.apache.accumulo.examples.simple.filedata.KeyUtilTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkInputStreamTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.373 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkCombinerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 sec

Results :

Tests run: 15, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ examples-simple ---
[INFO] Building jar: /usr/lib/accumulo/lib/examples-simple-1.4.3-cdh4.3.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] accumulo .......................................... SUCCESS [2.171s]
[INFO] cloudtrace ........................................ SUCCESS [3.827s]
[INFO] accumulo-start .................................... SUCCESS [20.561s]
[INFO] accumulo-core ..................................... SUCCESS [59.326s]
[INFO] accumulo-server ................................... SUCCESS [1:50.049s]
[INFO] accumulo-examples ................................. SUCCESS [0.016s]
[INFO] examples-simple ................................... SUCCESS [7.369s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:24.868s
[INFO] Finished at: Wed Feb 12 19:53:11 PST 2014
[INFO] Final Memory: 66M/168M
[INFO] ------------------------------------------------------------------------
[accumulo@localhost accumulo]$

Can you try doing the same via the commandline in your VM? If we see a failure there, it should be easier to chase down the cause.

[#9]

Can do!

R there any flags you what me to use during the build?

thanks!

Chris

[#10]

you should use the same flags I used in my example.