Member since
04-24-2016
4
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1731 | 04-26-2016 02:36 AM |
05-05-2016
02:19 AM
Default Spark env config error in HDP2.4 installation: #SPARK_DRIVER_MEMORY="512 Mb" This line include a wrong config param "512 Mb", it should be "512M" without space. This will cause spark-shell yarn client start error: Invalid initial heap size. Suggest to update the template.
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
04-26-2016
02:36 AM
Resolved. Need to edit the spark-env configure file to enable spark yarn client mode.
... View more
04-24-2016
01:57 PM
Hi, In HDP sandbox 2.4, When to start Spark with ./bin/spark-shell --master yarn --deploy-mode client After create a job, there's error: 16/04/24 13:42:24 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
... View more
Labels:
- Labels:
-
Apache Spark
04-24-2016
11:55 AM
If I want to run from external of the VM, for example, in Eclipse, what's the master ip and port? Thanks.
... View more