Member since
01-18-2018
2
Posts
0
Kudos Received
0
Solutions
01-19-2018
02:33 PM
This is the curl command: curl --user "abc:abc" -H "Content-Type: application/json" -X POST -d '{"className": "aa.aa.LivyWriteHive1","file": "/user/vivek.b/livy_vivek/LivyWriteHive1.jar"}' -i http://xxx.xxx.xxx.xxx:8998/batches
... View more
01-19-2018
02:32 PM
We have just installed Livy, not compiled, just used the binaries, unzip it , and now using.It is working with helo livy.But when trying to submit spark job, it is not working.logs... Warning: Skip remote jar hdfs://xxx.xxx.xxx.xxx:8020/user/vivek.b/livy_vivek/LivyWriteHive1.jar.
java.lang.ClassNotFoundException: com.scala.infy.LivyWriteHive1
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:175)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$runMain(SparkSubmit.scala:689)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) I think it is not able to read the HDFS. Can anybody help if some property need to set or something like that?
... View more
Labels:
- Labels:
-
Apache Spark