Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest

avatar
Explorer
Greetings to all esteem colleague,
 
Following indicate a build failure for accumulo-examples indicating  the failure of Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.
 
I would very much appreciate advice on actions to take in detail that would resolve this issue.
 
The environment is accumulo-1.4.3, CDH4.3 QuickStart using netbeans to import the maven project  accumulo-examples after installing accumulo from the accumulo-1.4.3cdh4.3.tar.gz.
 
The VM is available upon request.
 
Cloudera has not responded in there news groups.
 
In advance thanks for your assistance.
 
The results of executing the build with dependencies option are listed below:
 
cd /usr/lib/accumulo/src/examples/simple; JAVA_HOME=/usr/java/jdk1.6.0_32 /home/cloudera/netbeans-7.4/java/maven/bin/mvn install
Scanning for projects...
 
Some problems were encountered while building the effective model for org.apache.accumulo:examples-simple:jar:1.4.3-cdh4.3.0
'build.plugins.plugin.version' for org.apache.maven.plugins:maven-clean-plugin is missing. @ org.apache.accumulo:accumulo:1.4.3-cdh4.3.0, /usr/lib/accumulo/pom.xml, line 81, column 15
'build.plugins.plugin.version' for org.apache.maven.plugins:maven-resources-plugin is missing. @ org.apache.accumulo:accumulo:1.4.3-cdh4.3.0, /usr/lib/accumulo/pom.xml, line 152, column 15
 
It is highly recommended to fix these problems because they threaten the stability of your build.
 
For this reason, future Maven versions might no longer support building such malformed projects.
 
                                                                        
------------------------------------------------------------------------
Building examples-simple 1.4.3-cdh4.3.0
------------------------------------------------------------------------
 
--- maven-enforcer-plugin:1.0:enforce (enforce-mvn) @ examples-simple ---
 
--- maven-resources-plugin:2.5:resources (default-resources) @ examples-simple ---
[debug] execute contextualize
Using 'UTF-8' encoding to copy filtered resources.
skip non existing resourceDirectory /usr/lib/accumulo/src/examples/simple/src/main/resources
 
--- maven-dependency-plugin:2.1:copy-dependencies (copy-dependencies) @ examples-simple ---
log4j-1.2.16.jar already exists in destination.
 
--- maven-compiler-plugin:2.3.2:compile (default-compile) @ examples-simple ---
Compiling 40 source files to /usr/lib/accumulo/src/examples/simple/target/classes
 
--- maven-resources-plugin:2.5:testResources (default-testResources) @ examples-simple ---
[debug] execute contextualize
Using 'UTF-8' encoding to copy filtered resources.
skip non existing resourceDirectory /usr/lib/accumulo/src/examples/simple/src/test/resources
 
--- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ examples-simple ---
Compiling 5 source files to /usr/lib/accumulo/src/examples/simple/target/test-classes
 
--- maven-surefire-plugin:2.9:test (default-test) @ examples-simple ---
Surefire report directory: /usr/lib/accumulo/src/examples/simple/target/surefire-reports
 
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest
Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 1.89 sec <<< FAILURE!
Running org.apache.accumulo.examples.simple.filedata.ChunkCombinerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkInputStreamTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.175 sec
Running org.apache.accumulo.examples.simple.filedata.KeyUtilTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
Running org.apache.accumulo.examples.simple.dirlist.CountTest
Max depth              : 3
Time to find max depth : 2 ms
Time to compute counts : 7 ms
Entries scanned        : 30 
Counts inserted        : 4 
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.072 sec
 
Results :
 
Tests in error: 
  test(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest): Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
  testErrorOnNextWithoutClose(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest): Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
  testInfoWithoutChunks(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest): Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
 
Tests run: 15, Failures: 0, Errors: 3, Skipped: 0
 
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------
Total time: 17.620s
Finished at: Tue Feb 11 14:34:18 PST 2014
Final Memory: 20M/57M
------------------------------------------------------------------------
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.9:test (default-test) on project examples-simple: There are test failures.
 
Please refer to /usr/lib/accumulo/src/examples/simple/target/surefire-reports for the individual test results.
-> [Help 1]
 
To see the full stack trace of the errors, re-run Maven with the -e switch.
Re-run Maven using the -X switch to enable full debug logging.
 
For more information about the errors and possible solutions, please read the following articles:
-> [Help 1]
==============================================================================
 
Following is: The results of the failed report in  "/usr/lib/accumulo/src/examples/simple/target/surefire-reports"
 
-------------------------------------------------------------------------------
Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest
-------------------------------------------------------------------------------
Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 2.297 sec <<< FAILURE!
test(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest)  Time elapsed: 1.542 sec  <<< ERROR!
java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
        at org.apache.accumulo.core.client.mapreduce.InputFormatBase$RecordReaderBase.initialize(InputFormatBase.java:1108)
        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormat$1.initialize(ChunkInputFormat.java:47)
        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.test(ChunkInputFormatTest.java:95)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at junit.framework.TestCase.runTest(TestCase.java:168)
        at junit.framework.TestCase.runBare(TestCase.java:134)
        at junit.framework.TestResult$1.protect(TestResult.java:110)
        at junit.framework.TestResult.runProtected(TestResult.java:128)
        at junit.framework.TestResult.run(TestResult.java:113)
        at junit.framework.TestCase.run(TestCase.java:124)
        at junit.framework.TestSuite.runTest(TestSuite.java:232)
        at junit.framework.TestSuite.run(TestSuite.java:227)
        at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
testErrorOnNextWithoutClose(org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest)  Time elapsed: 0.084 sec  <<< ERROR!
java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
        at org.apache.accumulo.core.client.mapreduce.InputFormatBase$RecordReaderBase.initialize(InputFormatBase.java:1108)
        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormat$1.initialize(ChunkInputFormat.java:47)
        at org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.testErrorOnNextWithoutClose(ChunkInputFormatTest.java:146)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at junit.framework.TestCase.runTest(TestCase.java:168)
        at junit.framework.TestCase.runBare(TestCase.java:134)
        at junit.framework.TestResult$1.protect(TestResult.java:110)
        at junit.framework.TestResult.runProtected(TestResult.java:128)
        at junit.framework.TestResult.run(TestResult.java:113)
        at junit.framework.TestCase.run(TestCase.java:124)
        at junit.framework.TestSuite.runTest(TestSuite.java:232)
1 ACCEPTED SOLUTION

avatar
Explorer

Cleaned up as Root.

 

As accumulo executed:

 

Test results failed:mvn -Dhadoop.profile=2.0 package

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.accumulo.start.Test
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 11.703 sec <<< FAILURE!

Results :

Tests in error:
testChangingDirectory(org.apache.accumulo.start.Test): /usr/lib/accumulo/src/start/../../conf/examples/512MB/standalone/accumulo-site.xml (No such file or directory)

Tests run: 2, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] accumulo .......................................... SUCCESS [3.794s]
[INFO] cloudtrace ........................................ SUCCESS [11.847s]
[INFO] accumulo-start .................................... FAILURE [1:54.079s]
[INFO] accumulo-core ..................................... SKIPPED
[INFO] accumulo-server ................................... SKIPPED
[INFO] accumulo-examples ................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:11.538s
[INFO] Finished at: Tue Feb 18 23:24:34 PST 2014
[INFO] Final Memory: 34M/83M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.9:test (default-test) on project accumulo-start: There are test failures.
[ERROR]
[ERROR] Please refer to /usr/lib/accumulo/src/start/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :accumulo-start
[accumulo@localhost accumulo]$ whoami
accumulo
[accumulo@localhost accumulo]$

View solution in original post

18 REPLIES 18

avatar
Expert Contributor

Hi Chris!

 

This particular problem is caused by attempting to build Accumulo in Hadoop 1 mode against Hadoop 2 (specifically CDH4).

 

In the initial Cloudera integrated release of the Accumulo 1.4.x line, you still had to specify the Hadoop 2 profile via a Java system property.

 

Command line maven would look like this:

$> mvn -Dhadoop.profile=2.0 package

 I presume there is some way to specify this in the build options for netbeans, but I am not familiar with that particular IDE.

 

Alternatively, you could install our latest release, Accumulo 1.4.4-cdh4.5.0 (available from archive), which defaults to building against the Hadoop 2 profile.

 

It might also help to know why you're rebuilding the examples. From your other outstanding question, my guess is that you're using it as a starting point to learn about general development?

avatar
Explorer
Exactly!

I am new to netbeans as well and am not a Maven guy.

I'll see if I can use mvn -Dhadoop.profile=2.0 package.

I will be starting on my maven book in 20 minutes (sort of a machine
learning guy)

I was using the examples to be modified during training.

I will rebuild using accumulo Accumulo
1.4.4-cdh4.5.0
if
not.

Can I still use the current QuickStart VM?

appreciate your assistance Joey,

Chris

Ps. First response from Cloudera I have gotten with 4 posts



avatar
Guru
FYI, I have moved this thread to the Accumulo board...

avatar
Guru

@crigano I'm dropping your previous message into this thread, as well as @busbey 's since they both seem to have gotten derailed into an offline email thread.  and FWIW, there is no Joey on this thread, not sure if there is some threads getting crossed here or what.

 

From busbey:

 

Hi Chris!

If you're not already familiar with Netbeans, I'd recommend sticking with the development tools you're comfortable with.

You should check your notification settings; I know I responded to at least one of your other and I believe someone else responded to another.

-Sean

 

from crigano:

 

put the  in the IDE: mvn -Dhadoop.profile=2.0 package

 
Now I get completion in the IDE will not include classes from these dependencies or their transitive dependencies (unless they are among the open projects).
Please download the dependencies, or install them manually, if not available remotely.
 
The artifacts are:
 org.apache.hadoop:hadoop-core:jar:2.0.0-cdh4.3.0
 
What do you recommend Joey,
 
thanks,
 
Chris

 

 

avatar
Expert Contributor

Hi Chris!

 

That definitely sounds like a problem with getting Netbeans to play nicely with Maven's dependency providing.

 

To avoid the drag of these kinds of tooling issues, I generally recommend people stick with whatever workflow they're used to for Java development.

 

Is there another IDE you are more familiar with?

 

-Sean

avatar
Explorer
My Bad,

I set up notifications to go to chris.p.rigano@gmail.com.

I am just trying to get a version of the CDH QuickStart with accumulo to
run for classes I am teaching and to do work outside the IC doors.

I tried both eclipse I have been using for a number of years and got the
right setting in netbeans.

Still not working as indicated in literature for examples.

This is not an ide issue but an environment problem.

I would be glad to send you my VM built IAW cloudera guidance.

It may make things easier.

Give me access to a fast FTP.

I used CDH at Meade for a while. I am a cloudera fan.

Best regards,

Chris




avatar
Expert Contributor

Let's step back and confirm that your VM can properly see the necessary Maven repositories.

 

I'm using a QuickStart VM with CDH 4.4.0 and I have the Accumulo 1.4.3-cdh4.3.0 release installed from tarballs in /usr/lib/accumulo.

 

If I open up a terminal and navigate to the install directory, I can successfully rebuild everything including the examples.

 

[cloudera@localhost ~]$ sudo su - accumulo
[accumulo@localhost ~]$ cd /usr/lib/accumulo
[accumulo@localhost accumulo]$ mvn -Dhadoop.profile=2.0 package
<...snip maven downloading dependencies from the internet...>
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.accumulo.examples.simple.dirlist.CountTest
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Max depth              : 3
Time to find max depth : 7 ms
Time to compute counts : 7 ms
Entries scanned        : 30 
Counts inserted        : 4 
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.302 sec
Running org.apache.accumulo.examples.simple.filedata.KeyUtilTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkInputStreamTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.373 sec
Running org.apache.accumulo.examples.simple.filedata.ChunkCombinerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 sec

Results :

Tests run: 15, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ examples-simple ---
[INFO] Building jar: /usr/lib/accumulo/lib/examples-simple-1.4.3-cdh4.3.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] accumulo .......................................... SUCCESS [2.171s]
[INFO] cloudtrace ........................................ SUCCESS [3.827s]
[INFO] accumulo-start .................................... SUCCESS [20.561s]
[INFO] accumulo-core ..................................... SUCCESS [59.326s]
[INFO] accumulo-server ................................... SUCCESS [1:50.049s]
[INFO] accumulo-examples ................................. SUCCESS [0.016s]
[INFO] examples-simple ................................... SUCCESS [7.369s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:24.868s
[INFO] Finished at: Wed Feb 12 19:53:11 PST 2014
[INFO] Final Memory: 66M/168M
[INFO] ------------------------------------------------------------------------
[accumulo@localhost accumulo]$

 

Can you try doing the same via the commandline in your VM? If we see a failure there, it should be easier to chase down the cause.

avatar
Explorer

Can do!

 

R there any flags you what me to use during the build?

 

 

thanks!

 

Chris

avatar
Expert Contributor

you should use the same flags I used in my example.