Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Curl command to submit spark jar using Livy is not working

avatar
New Contributor

We have just installed Livy, not compiled, just used the binaries, unzip it , and now using.It is working with helo livy.But when trying to submit spark job, it is not working.logs...
Warning: Skip remote jar hdfs://xxx.xxx.xxx.xxx:8020/user/vivek.b/livy_vivek/LivyWriteHive1.jar. java.lang.ClassNotFoundException: com.scala.infy.LivyWriteHive1 at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.spark.util.Utils$.classForName(Utils.scala:175) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$runMain(SparkSubmit.scala:689) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

I think it is not able to read the HDFS.

Can anybody help if some property need to set or something like that?

2 REPLIES 2

avatar
New Contributor

This is the curl command:
curl --user "abc:abc" -H "Content-Type: application/json" -X POST -d '{"className": "aa.aa.LivyWriteHive1","file": "/user/vivek.b/livy_vivek/LivyWriteHive1.jar"}' -i http://xxx.xxx.xxx.xxx:8998/batches

avatar
New Contributor

M stuck with the same error.Where you able to find a solution