Member since
12-10-2015
5
Posts
3
Kudos Received
0
Solutions
04-13-2020
12:25 PM
Hi, can I instead add the following line to spark-defaults.conf file: spark.ui.port 4041 Will that have the same effect ? Thanks
... View more
02-16-2016
07:45 PM
@hoda moradi http://spark.apache.org/docs/latest/streaming-kafka-integration.html It has java code
... View more
03-30-2016
09:58 AM
1 Kudo
I have the same issue. I tried setting this parameter in spark-defaults.conf as well passed along spark-submit (spark-submit file.py --conf spark.shuffle.consolidateFiles=true). Still getting the same error.
... View more
02-02-2016
01:59 AM
@Lakumi Narayanan can you accept the best answer to close this thread?
... View more
12-11-2015
10:36 PM
Additionally, if you want to change number of partitions (and then parallelism) of an existing RDD, you can use rdd.repartition(8) See the comments and tests from here:
https://community.hortonworks.com/questions/5825/best-way-to-select-distinct-values-from-multiple-c.html
... View more