Reply
New Contributor
Posts: 4
Registered: ‎07-20-2017

Error while working with csv file in Envelope

i am using following conf file

 

application {
name = VoteData
executors = 2
}
steps {
fsInput {
input {
type = filesystem
path =/user/prashanth/Vote.txt
format = csv
separator ="\t"
field.names = [data1,data2,data3,data4,date]
field.types = [int,int,double,double,string]
}
}
fsProcess{
dependencies = [fsInput]
deriver {
type = sql
query.literal = "SELECT * FROM fsInput"
}
planner = {
type = append
}
output = {
type = filesystem
path = /user/prashanth
format = csv
}
}
}

 

 

i am getting the following error

 

Exception in thread "main" com.typesafe.config.ConfigException$BadPath: path parameter: Invalid path 'null-value': Token not allowed in path expression: '-' (Invalid number: '-') (you can double-quote this token if you really want it here)
at com.typesafe.config.impl.Parser.parsePathExpression(Parser.java:934)

 

 

Cloudera Employee
Posts: 34
Registered: ‎08-26-2015

Re: Error while working with csv file in Envelope

Hi,

I tried to replicate the error but wasn't able to. Are you sure that was
the configuration file you used? I don't see any reference to 'null-value'
in there. But either way if you could provide the full stack trace that
should be enough to narrow it down.
New Contributor
Posts: 4
Registered: ‎07-20-2017

Re: Error while working with csv file in Envelope

Exception in thread "main" com.typesafe.config.ConfigException$BadPath: path parameter: Invalid path 'null-value': Token not allowed in path expression: '-' (Invalid number: '-') (you can double-quote this token if you really want it here)
at com.typesafe.config.impl.Parser.parsePathExpression(Parser.java:934)
at com.typesafe.config.impl.Parser.parsePath(Parser.java:974)
at com.typesafe.config.impl.Path.newPath(Path.java:220)
at com.typesafe.config.impl.SimpleConfig.hasPath(SimpleConfig.java:71)
at com.cloudera.labs.envelope.utils.ConfigUtils$OptionMap.resolve(ConfigUtils.java:80)
at com.cloudera.labs.envelope.input.FileSystemInput.configure(FileSystemInput.java:165)
at com.cloudera.labs.envelope.input.InputFactory.create(InputFactory.java:62)
at com.cloudera.labs.envelope.run.Runner.extractSteps(Runner.java:114)
at com.cloudera.labs.envelope.run.Runner.run(Runner.java:72)
at com.cloudera.labs.envelope.EnvelopeMain.main(EnvelopeMain.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Cloudera Employee
Posts: 34
Registered: ‎08-26-2015

Re: Error while working with csv file in Envelope

Interesting. It appears Envelope is referencing Typesafe Config v1.0.2
instead of the v1.2.1 that it is supposed to compile with. Can you provide:

- How you built the Envelope jar
- Your spark2-submit call
- Your CDH and Spark versions
Explorer
Posts: 6
Registered: ‎09-26-2018

Re: Error while working with csv file in Envelope

Hi Jeremy,

We spoke briefly yesterday on Sentara's Envelope-Kudu call. I maven "package"d the envelope project to generate the jar files in the build folder(and target folder of the external sources) using IntelliJ's Maven(v3.3.9) plugin on Window.I ran into similar problems when trying out the filesystem example. Apparently, jar files created on Unix environment are working. We're on CDH15.5 and spark version 2.3.0.cloudera3
 

 

[rxsubram@edpdev-b4bbb72c envelope-master]$ spark2-submit /opt/cloudera/scripts/envelope-master/*.jar examples/filesystem/filesystem.conf
18/12/04 22:25:31 INFO envelope.EnvelopeMain: Envelope application started
Exception in thread "main" com.typesafe.config.ConfigException$Parse: /opt/cloudera/scripts/envelope-master/envelope-hbase-0.6.0.jar: 2: Key '"PK\u0003\u0004\u0014\u0000\b\b\b\u0000���M\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0014\u0000\u0004\u0000META-INF/MANIFEST"."MF��\u0000\u0000E��"' may not be followed by token: '�0�|C' (if you intended '�0�|C' to be part of the value for '"PK\u0003\u0004\u0014\u0000\b\b\b\u0000���M\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0014\u0000\u0004\u0000META-INF/MANIFEST"."MF��\u0000\u0000E��"', try enclosing the value in double quotes)
at com.typesafe.config.impl.Parser$ParseContext.parseError(Parser.java:435)
at com.typesafe.config.impl.Parser$ParseContext.parseError(Parser.java:431)
at com.typesafe.config.impl.Parser$ParseContext.parseObject(Parser.java:786)
at com.typesafe.config.impl.Parser$ParseContext.parse(Parser.java:993)
at com.typesafe.config.impl.Parser.parse(Parser.java:34)
at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:222)
at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:210)
at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:176)
at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:170)
at com.typesafe.config.impl.Parseable.parse(Parseable.java:227)
at com.typesafe.config.ConfigFactory.parseFile(ConfigFactory.java:595)
at com.typesafe.config.ConfigFactory.parseFile(ConfigFactory.java:599)
at com.cloudera.labs.envelope.utils.ConfigUtils.configFromPath(ConfigUtils.java:38)
at com.cloudera.labs.envelope.EnvelopeMain.main(EnvelopeMain.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
18/12/04 22:25:32 INFO util.ShutdownHookManager: Shutdown hook called
18/12/04 22:25:32 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-dc509fa3-d861-424c-9327-3f17523cb232

 

 

I also observed that when I build envelope, both 1.0.2 and 1.2.1 jar versions of typesafe config are downloaded. However, I couldn't get the dependency tree for you. the lib module's build breaks (I'm on Windows without setting up the needed services, of course!) Do let me know if I can get this setup done somehow on Windows and I'd be happy to debug with you.

 

Warm Regards,

Rema S

Cloudera Employee
Posts: 34
Registered: ‎08-26-2015

Re: Error while working with csv file in Envelope

Hi Rema, could you please start this in a new thread? Your issue does not look related to the original topic.

Jeremy
Explorer
Posts: 6
Registered: ‎09-26-2018

Re: Error while working with csv file in Envelope

Sure, did it.

 

Thanks!

Rema S

Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB