Member since
01-14-2015
23
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
43588 | 02-15-2017 08:14 AM |
02-15-2017
08:14 AM
It seems to be a library conflict issue between open source Spark 1.6.1 and Cloudera's Spark. I changed my POM file to use Spark version :- 1.6.0-cdh5.9.1 And now it is working fine. P.S :- If you run into the following error you might have "spark.shuffle.encryption.enabled" set to true. Caused by: java.lang.NullPointerException at com.intel.chimera.stream.CryptoOutputStream.(CryptoOutputStream.java:124) at com.intel.chimera.stream.CryptoOutputStream.(CryptoOutputStream.java:113) at com.intel.chimera.stream.CryptoOutputStream.(CryptoOutputStream.java:102) at com.intel.chimera.stream.CryptoOutputStream.(CryptoOutputStream.java:89) at org.apache.spark.crypto.CryptoStreamUtils$.createCryptoOutputStream(CryptoStreamUtils.scala:51) at org.apache.spark.storage.DiskBlockObjectWriter.open(DiskBlockObjectWriter.scala:104) at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
... View more
03-19-2015
07:52 PM
Apache just released 1.3.0 and I would like to install that on my CDH cluster. I know it will added to 5.4 CDH release but is there any other way to get it now? Or any other way to upgrade the existing SPARK standalone installation from 1.2.0 to 1.3.0.
... View more
01-15-2015
02:33 PM
Actually, there is a safety valve for Hive clients in the Hive service. This property should be specified there and it will work as Hue will get it:
hive.server2.authentication
NOSASL
I am forwarding this to the VM guys so that they fix it for next time!
... View more