Created 02-06-2019 01:56 PM
HI
I have the following entry in Pom.xml. and when i run the mvn clean compile i get the error as below.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.cloudera.labs</groupId>
<version>0.1.1</version>
<packaging>jar</packaging>
<name>inversion</name>
<dependencies>
<dependency>
<groupId>com.cloudera.labs</groupId>
<artifactId>envelope</artifactId>
<version>0.1.1</version>
</dependency>
<!-- project specific -->
Could not resolve dependencies for project com.cloudera.labs:luminainversion:jar:0.1.1: Failure to find com.cloudera.labs:envelope:jar:0.1.1 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
<br>
I found that envelope is not available in Maven Central. Can you please help me where i can find latest envelope jar ?
Created 02-12-2019 12:13 PM
@Jeremy Beard wrote:
Ok yes so the wildcard is expanding out to both jar filenames, and so the
second argument becomes the second jar and not "traffic.conf". Instead you
just specify the one jar filename.
Okay. Thanks
Makins some progress with you help. i am able to move to next issue
I am running Kafka_2.11-0.10.1.1
C:\source\envelope\examples\target>spark-submit2 envelope-0.6.0-shaded.jar traffic.conf
Exception in thread "main" java.lang.RuntimeException: Kafka input offset management not currently compatible with stream windowing.
at com.cloudera.labs.envelope.kafka.KafkaInput.configure(KafkaInput.java:102)
at com.cloudera.labs.envelope.input.InputFactory.create(InputFactory.java:39)
at com.cloudera.labs.envelope.run.DataStep.getInput(DataStep.java:166)
at com.cloudera.labs.envelope.run.DataStep.getComponents(DataStep.java:357)
at com.cloudera.labs.envelope.run.StreamingStep.getComponents(StreamingStep.java:122)
at com.cloudera.labs.envelope.run.Runner.initializeSecurity(Runner.java:481)
at com.cloudera.labs.envelope.run.Runner.run(Runner.java:99)
at com.cloudera.labs.envelope.EnvelopeMain.main(EnvelopeMain.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Created 02-12-2019 12:19 PM
Created 02-07-2019 07:33 AM
Hi Shrini,
We do not currently have the Envelope binaries on Maven Central but is something we are hoping to do in the future.
Just to understand what you're trying to do, are you looking to build a data pipeline on Envelope, or are you doing something more advanced like developing your own Envelope plugin?
If you are building a data pipeline on Envelope then you should just "mvn package" the Envelope project, and then use Spark to run the Envelope jar that is created, as per the readme and the examples.
If you are developing your own Envelope plugin then you should first "mvn install" the Envelope project on your machine so that the Envelope jar is installed in your local Maven repository, and then you will be able to use it in the pom of another project. Note that the version of Envelope on the default 'master' branch is 0.6.0, not 0.1.1.
Jeremy
Created 02-07-2019 08:25 AM
Thanks for your response.
I got the latest from github and ran mvn clean package
And i am getting compilation errrors. Any idea why ?
Thanks in advance.
Downloading: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.jar
Downloading: https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-junit/2.0.0.0/hamcrest-junit-2.0.0.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/hamcrest/java-hamcrest/2.0.0.0/java-hamcrest-2.0.0.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/jmockit/jmockit/1.22/jmockit-1.22.jar
Downloaded: https://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-junit/2.0.0.0/hamcrest-junit-2.0.0.0.jar (14 kB at 184 kB/s)
Downloaded: https://repo.maven.apache.org/maven2/org/hamcrest/java-hamcrest/2.0.0.0/java-hamcrest-2.0.0.0.jar (113 kB at 98 kB/s)
Downloaded: https://repo.maven.apache.org/maven2/org/jmockit/jmockit/1.22/jmockit-1.22.jar (615 kB at 140 kB/s)
Downloaded: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.jar (2.2 MB at 283 kB/s)
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ envelope-core ---
[INFO]
[INFO] --- license-maven-plugin:3.0:format (format) @ envelope-core ---
[INFO] Updating license headers...
[INFO]
[INFO] --- asciidoctor-maven-plugin:1.5.6:process-asciidoc (output-html) @ envelope-core ---
[INFO] sourceDirectory C:\source\envelope\core\docs does not exist. Skip processing
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ envelope-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\source\envelope\core\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.1:compile (default-compile) @ envelope-core ---
[INFO] Compiling 77 source files to C:\source\envelope\core\target\classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-core_2.11\2.1.0\spark-core_2.11-2.1.0.jar; invalid LOC header (bad sign
ature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\avro\avro-ipc\1.7.7\avro-ipc-1.7.7.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\com\twitter\chill_2.11\0.8.0\chill_2.11-0.8.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-network-common_2.11\2.1.0\spark-network-common_2.11-2.1.0.jar; invalid
LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\fusesource\leveldbjni\leveldbjni-all\1.8\leveldbjni-all-1.8.jar; invalid LOC header (bad s
ignature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\commons\commons-math3\3.4.1\commons-math3-3.4.1.jar; invalid CEN header (bad signat
ure)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scala-lang\scala-library\2.11.8\scala-library-2.11.8.jar; invalid LOC header (bad signatur
e)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\json4s\json4s-core_2.11\3.2.11\json4s-core_2.11-3.2.11.jar; invalid CEN header (bad signat
ure)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scala-lang\scalap\2.11.0\scalap-2.11.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\io\netty\netty-all\4.0.42.Final\netty-all-4.0.42.Final.jar; invalid CEN header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\io\netty\netty\3.8.0.Final\netty-3.8.0.Final.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scala-lang\scala-reflect\2.11.7\scala-reflect-2.11.7.jar; invalid LOC header (bad signatur
e)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\ivy\ivy\2.4.0\ivy-2.4.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\oro\oro\2.0.8\oro-2.0.8.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\net\razorvine\pyrolite\4.13\pyrolite-4.13.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\scalatest\scalatest_2.11\2.2.6\scalatest_2.11-2.2.6.jar; invalid LOC header (bad signature
)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-sql_2.11\2.1.0\spark-sql_2.11-2.1.0.jar; invalid LOC header (bad signat
ure)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\spark\spark-catalyst_2.11\2.1.0\spark-catalyst_2.11-2.1.0.jar; invalid CEN header (
bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\codehaus\janino\janino\3.0.0\janino-3.0.0.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\antlr\antlr4-runtime\4.5.3\antlr4-runtime-4.5.3.jar; invalid LOC header (bad signature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\parquet\parquet-column\1.8.1\parquet-column-1.8.1.jar; invalid CEN header (bad sign
ature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\org\apache\parquet\parquet-jackson\1.8.1\parquet-jackson-1.8.1.jar; invalid LOC header (bad si
gnature)
[ERROR] error: error reading C:\Users\hb67410\.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar; invalid LOC header (bad signature)
[INFO] 23 errors
Created 02-07-2019 09:04 AM
It looks like you have corrupted files in your local Maven repository. You might need to remove your .m2 directory and then Maven will re-download the dependencies when you try to compile Envelope again.
Created 02-07-2019 09:05 AM
I should have mentioned earlier that we do have the Envelope binaries on the GitHub Releases page: https://github.com/cloudera-labs/envelope/releases. This might be easier if you are having trouble compiling the project.
Created 02-12-2019 08:34 AM
hi
Thanks,Thats great help
Created 02-12-2019 08:55 AM
Hi
Thanks for your note.
I was able to finally build and compile. But now when i try to run the traffic example i am getting this error
Exception in thread "main" com.typesafe.config.ConfigException$Parse: envelope-examples-0.6.0-shaded.jar: 2: Key '"PK\u0003\u0004\u0014\u0000\b\b\b\u0000?t
KN\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0014\u0000\u0004\u0000META-INF/MANIFEST"."MF??\u0000\u0000E??"' may not be foll
owed by token: '?0►E???C~' (if you intended '?0►E???C~' to be part of the value for '"PK\u0003\u0004\u0014\u0000\b\b\b\u0000?tKN\u0000\u0000\u0000\u0000\u0
000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0014\u0000\u0004\u0000META-INF/MANIFEST"."MF??\u0000\u0000E??"', try enclosing the value in double quotes)
at com.typesafe.config.impl.Parser$ParseContext.parseError(Parser.java:435)
at com.typesafe.config.impl.Parser$ParseContext.parseError(Parser.java:431)
at com.typesafe.config.impl.Parser$ParseContext.parseObject(Parser.java:786)
at com.typesafe.config.impl.Parser$ParseContext.parse(Parser.java:993)
Anyh idea?
Created 02-12-2019 10:34 AM
Created 02-12-2019 11:57 AM
hi
The command is
C:\source\envelope\examples\target>spark-submit2 envelope-*.jar traffic.conf
These are the two jars i have in the folder C:\source\envelope\examples\target
envelope-0.6.0-shaded.jar
envelope-examples-0.6.0-shaded.jar
Thanks
Created 02-12-2019 12:06 PM