Reply
New Contributor
Posts: 1
Registered: ‎05-24-2018

json case some match is working fine in spark shell and not in spark-submit

 

 please find the below code:

 

"result" is the type of scala.Some

 

result match {
case Some(map: Map[String, Any]) => {
println(" block one::")
val error = map("error")
println("error-->" + error)

 

 

the above code is working fine in spark-shell but not working using spark-submit with jar file.

spark-submit --driver-memory 3G --executor-memory 3G --num-executors 3 --executor-cores 3 --class com.hcl.spark.SampleHttp --master yarn-cluster /home/eih/prod/usermdm/jars/code/spark_ad_ingestion.jar 

 

Appreciate your help as soon as possible

 

 

Cloudera Employee
Posts: 8
Registered: ‎05-15-2018

Re: json case some match is working fine in spark shell and not in spark-submit

Hello Sreeja,

 

Thank you for posting your query

 

Could you please pass some addtional information related to this issue ?

 

Example.

1. Error message which you are seeing

2. How you are compiling / executing this code etc.,

 

Thanks,

Satz

Thanks,
Satz
Announcements