question Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ? in Support Questions

STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

keerthanakumar2 — Sun, 07 Feb 2016 08:26:09 GMT

I'm getting this error :

The POM for com.google.guava:guava-testlib:jar:16.0.1 is invalid, transitive dependencies (if any) will not be available: 1 problem was encountered while building the effective model for com.google.guava:guava-testlib:[unknown-version]

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

keerthanakumar2 — Sun, 07 Feb 2016 08:26:57 GMT

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project storm-elasticsearch: Compilation failure: Compilation failure:
[ERROR] /root/storm/external/storm-elasticsearch/src/test/java/org/apache/storm/elasticsearch/common/EsConfigTest.java:[23,33] package com.google.common.collect does not exist
[ERROR] /root/storm/external/storm-elasticsearch/src/test/java/org/apache/storm/elasticsearch/common/EsConfigTest.java:[23,33] package com.google.common.collect does not exist
[ERROR] /root/storm/external/storm-elasticsearch/src/test/java/org/apache/storm/elasticsearch/common/EsConfigTest.java:[57,16] cannot find symbol
[ERROR] symbol:   variable ImmutableMap
[ERROR] location: class org.apache.storm.elasticsearch.common.EsConfigTest
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project storm-elasticsearch: Compilation failure
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:160)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
        at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:858)
        at org.apache.maven.plugin.compiler.TestCompilerMojo.execute(TestCompilerMojo.java:152)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        ... 19 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :storm-elasticsearch

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

aervits — Sun, 07 Feb 2016 08:43:57 GMT

@keerthana gajarajakumar

what version of maven do you have and please explain what you're doing to reproduce.

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

keerthanakumar2 — Sun, 07 Feb 2016 09:30:02 GMT

Maven version is 3.2.5. I'm trying to execute storm-starter ExclamationTopology.

storm jar ./storm/examples/storm-starter/target/storm-starter-2.0.0-SNAPSHOT.jar org.apache.storm.starter.ExclamationTopology

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

phargis — Sun, 07 Feb 2016 09:38:10 GMT

Leave storm/lib vanilla and package the higher versions of those libs with your topology jar using maven shade to relocate the necessary packages to avoid conflict. Here is an example from a storm topology that relocates guava (com.google.common).

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>${version.shade}</version>
    <configuration>
        <relocations>
            <relocation>
                <pattern>com.google.common</pattern>
                <shadedPattern>com.cisco.com.google.common</shadedPattern>
            </relocation>
        </relocations>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <filters>
                    <filter>
                        <artifact>*:*</artifact>
                        <excludes>
                            <exclude>META-INF/*.SF</exclude>
                            <exclude>META-INF/*.DSA</exclude>
                            <exclude>META-INF/*.RSA</exclude>
                        </excludes>
                    </filter>
                </filters>
                <artifactSet>
                    <excludes>
                        <exclude>org.datanucleus</exclude>
                    </excludes>
                </artifactSet>
                <promoteTransitiveDependencies>true</promoteTransitiveDependencies>
                <transformers>
                    <transformer
                            implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"
/>
                </transformers>
            </configuration>
        </execution>
    </executions>
</plugin>

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

keerthanakumar2 — Sun, 07 Feb 2016 09:51:45 GMT

@Paul Hargis -- Thanks for your time and suggestions. I was able to solve the guava dependency issue. Now I'm getting another error due to external Kafka dependency. Can you guide me?

Running org.apache.storm.kafka.bolt.KafkaBoltTest Tests run: 16, Failures: 0, Errors: 16, Skipped: 0, Time elapsed: 9.338 sec <<< FAILURE! - in org.apache.storm.kafka.bolt.KafkaBoltTest executeWithBrokerDown(org.apache.storm.kafka.bolt.KafkaBoltTest) Time elapsed: 1.65 sec <<< ERROR!

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

amcbarnett — Sun, 07 Feb 2016 11:22:37 GMT

@keerthana gajarajakumar please post this as a new question on HCC

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

keerthanakumar2 — Sun, 07 Feb 2016 11:41:17 GMT

@Ancil McBarnett -- Hey I made this a seperate question. Thanks!

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

public_charles — Tue, 23 Feb 2016 13:22:29 GMT

@Paul Hargis Follow up question. I have a topology that used both hBase and ElasticSearch 2.1. hBase depends on guava 1.3 and ElasticSearch 2.1 depends on guava 1.8. Can I include both versions of guava? I somehow need to tell maven to rewrite the Elastic byte code to refer to the relocated guava 1.8, but to NOT rewrite hBase byte code, so it continues to find guava 1.3.

Is this possible? How?

( It seems I could do this by breaking up my project into more pom files and isolating them that way, but that will break the more functional segmentation of my multi-pom project, so maybe there is a more direct way?)

Re: STORM : How do I fix the google.guava dependency while running mvn clean install -DskipTests=true ?

aervits — Tue, 23 Feb 2016 19:40:02 GMT

You can exclude certain sub dependencies with <exclude> tag, read the maven docs for that