Maven : Dowenloading Envelope binaries

[#1]

HI

I have the following entry in Pom.xml. and when i run the mvn clean compile i get the error as below.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.cloudera.labs</groupId>
<version>0.1.1</version>
<packaging>jar</packaging>

<name>inversion</name>



<dependencies>

<dependency>
<groupId>com.cloudera.labs</groupId>
<artifactId>envelope</artifactId>
<version>0.1.1</version>
</dependency>

<!-- project specific -->

Could not resolve dependencies for project com.cloudera.labs:luminainversion:jar:0.1.1: Failure to find com.cloudera.labs:envelope:jar:0.1.1 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

<br>

I found that envelope is not available in Maven Central. Can you please help me where i can find latest envelope jar ?

[#2]

Hi Shrini,

We do not currently have the Envelope binaries on Maven Central but is something we are hoping to do in the future.

Just to understand what you're trying to do, are you looking to build a data pipeline on Envelope, or are you doing something more advanced like developing your own Envelope plugin?

If you are building a data pipeline on Envelope then you should just "mvn package" the Envelope project, and then use Spark to run the Envelope jar that is created, as per the readme and the examples.

If you are developing your own Envelope plugin then you should first "mvn install" the Envelope project on your machine so that the Envelope jar is installed in your local Maven repository, and then you will be able to use it in the pom of another project. Note that the version of Envelope on the default 'master' branch is 0.6.0, not 0.1.1.

Jeremy

[#3]

Thanks for your response. 

I got the latest from github and ran mvn clean package

And i am getting compilation errrors. Any idea why ?

Thanks in advance. 

Downloading: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.jar
Downloading: https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-junit/2.0.0.0/hamcrest-junit-2.0.0.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/hamcrest/java-hamcrest/2.0.0.0/java-hamcrest-2.0.0.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/jmockit/jmockit/1.22/jmockit-1.22.jar
Downloaded: https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-junit/2.0.0.0/hamcrest-junit-2.0.0.0.jar (14 kB at 184 kB/s)
Downloaded: https://repo.maven.apache.org/maven2/org/hamcrest/java-hamcrest/2.0.0.0/java-hamcrest-2.0.0.0.jar (113 kB at 98 kB/s)
Downloaded: https://repo.maven.apache.org/maven2/org/jmockit/jmockit/1.22/jmockit-1.22.jar (615 kB at 140 kB/s)
Downloaded: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.jar (2.2 MB at 283 kB/s)
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ envelope-core ---
[INFO]
[INFO] --- license-maven-plugin:3.0:format (format) @ envelope-core ---
[INFO] Updating license headers...
[INFO]
[INFO] --- asciidoctor-maven-plugin:1.5.6:process-asciidoc (output-html) @ envelope-core ---
[INFO] sourceDirectory C:\source\envelope\core\docs does not exist. Skip processing
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ envelope-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\source\envelope\core\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.1:compile (default-compile) @ envelope-core ---
[INFO] Compiling 77 source files to C:\source\envelope\core\target\classes
[INFO] -------------------------------------------------------------

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-core_2.11\2.1.0\spark-core_2.11-2.1.0.jar; invalid LOC header (bad sign
ature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\avro\avro-ipc\1.7.7\avro-ipc-1.7.7.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\com\twitter\chill_2.11\0.8.0\chill_2.11-0.8.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-network-common_2.11\2.1.0\spark-network-common_2.11-2.1.0.jar; invalid
LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\fusesource\leveldbjni\leveldbjni-all\1.8\leveldbjni-all-1.8.jar; invalid LOC header (bad s
ignature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\commons\commons-math3\3.4.1\commons-math3-3.4.1.jar; invalid CEN header (bad signat
ure)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scala-lang\scala-library\2.11.8\scala-library-2.11.8.jar; invalid LOC header (bad signatur
e)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\json4s\json4s-core_2.11\3.2.11\json4s-core_2.11-3.2.11.jar; invalid CEN header (bad signat
ure)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scala-lang\scalap\2.11.0\scalap-2.11.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\io\netty\netty-all\4.0.42.Final\netty-all-4.0.42.Final.jar; invalid CEN header (bad signature)

[ERROR] error: error reading C:\Users\hb67410\.m2\repository\io\netty\netty\3.8.0.Final\netty-3.8.0.Final.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scala-lang\scala-reflect\2.11.7\scala-reflect-2.11.7.jar; invalid LOC header (bad signatur
e)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\ivy\ivy\2.4.0\ivy-2.4.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\oro\oro\2.0.8\oro-2.0.8.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\net\razorvine\pyrolite\4.13\pyrolite-4.13.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scalatest\scalatest_2.11\2.2.6\scalatest_2.11-2.2.6.jar; invalid LOC header (bad signature
)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-sql_2.11\2.1.0\spark-sql_2.11-2.1.0.jar; invalid LOC header (bad signat
ure)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-catalyst_2.11\2.1.0\spark-catalyst_2.11-2.1.0.jar; invalid CEN header (
bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\codehaus\janino\janino\3.0.0\janino-3.0.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\antlr\antlr4-runtime\4.5.3\antlr4-runtime-4.5.3.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\parquet\parquet-column\1.8.1\parquet-column-1.8.1.jar; invalid CEN header (bad sign
ature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\parquet\parquet-jackson\1.8.1\parquet-jackson-1.8.1.jar; invalid LOC header (bad si
gnature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar; invalid LOC header (bad signature)
[INFO] 23 errors

[#4]

It looks like you have corrupted files in your local Maven repository. You might need to remove your .m2 directory and then Maven will re-download the dependencies when you try to compile Envelope again.

[#5]

I should have mentioned earlier that we do have the Envelope binaries on the GitHub Releases page: https://github.com/cloudera-labs/envelope/releases. This might be easier if you are having trouble compiling the project.

[#6]

hi

Thanks,Thats great help

[#7]

Hi 

Thanks for your note.

I was able to finally build and compile. But now when i try to run the traffic example i am getting this error 

Exception in thread "main" com.typesafe.config.ConfigException$Parse: envelope-examples-0.6.0-shaded.jar: 2: Key '"PK\u0003\u0004\u0014\u0000\b\b\b\u0000?t
KN\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0014\u0000\u0004\u0000META-INF/MANIFEST"."MF??\u0000\u0000E??"' may not be foll
owed by token: '?0►E???C~' (if you intended '?0►E???C~' to be part of the value for '"PK\u0003\u0004\u0014\u0000\b\b\b\u0000?tKN\u0000\u0000\u0000\u0000\u0
000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0014\u0000\u0004\u0000META-INF/MANIFEST"."MF??\u0000\u0000E??"', try enclosing the value in double quotes)
at com.typesafe.config.impl.Parser$ParseContext.parseError(Parser.java:435)
at com.typesafe.config.impl.Parser$ParseContext.parseError(Parser.java:431)
at com.typesafe.config.impl.Parser$ParseContext.parseObject(Parser.java:786)
at com.typesafe.config.impl.Parser$ParseContext.parse(Parser.java:993)

Anyh idea?

[#8]

It looks like you have provided the Envelope jar where the configuration
file is supposed to be provided. Can you show what your spark2-submit (or
maybe spark-submit) call looks like?

[#9]

hi

The command is

C:\source\envelope\examples\target>spark-submit2 envelope-*.jar traffic.conf

These are the two jars i have in the folder C:\source\envelope\examples\target

envelope-0.6.0-shaded.jar
envelope-examples-0.6.0-shaded.jar

Thanks

[#10]

Ok yes so the wildcard is expanding out to both jar filenames, and so the
second argument becomes the second jar and not "traffic.conf". Instead you
just specify the one jar filename.