<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question Re: pyspark crashes when running locally but works on a cluster in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/pyspark-crashes-when-running-locally-but-works-on-a-cluster/m-p/51725#M33389</link>
    <description>&lt;P&gt;Same applies for spark-submit. It works locally but not on the cluster:&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;spark-submit --master=yarn --deploy-mode=client lettercount.py&lt;BR /&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;works but&amp;nbsp;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;spark-submit --master=yarn --deploy-mode=cluster lettercount.py&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;crashes:&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;========&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;...&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;client token: N/A&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;diagnostics: Application application_1488351192359_0019 failed 2 times due to AM Container for appattempt_1488351192359_0019_000002 exited with &amp;nbsp;exitCode: 1&lt;BR /&gt;For more detailed output, check application tracking page:&lt;A href="http://md02.rcc.local:8088/proxy/application_1488351192359_0019/Then" target="_blank"&gt;http://md02.rcc.local:8088/proxy/application_1488351192359_0019/Then&lt;/A&gt;, click on links to logs of each attempt.&lt;BR /&gt;Diagnostics: Exception from container-launch.&lt;BR /&gt;Container id: container_1488351192359_0019_02_000001&lt;BR /&gt;Exit code: 1&lt;BR /&gt;Stack trace: ExitCodeException exitCode=1: &amp;nbsp;&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at org.apache.hadoop.util.Shell.runCommand(Shell.java:601)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at org.apache.hadoop.util.Shell.run(Shell.java:504)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:786)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at java.util.concurrent.FutureTask.run(FutureTask.java:266)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)&lt;BR /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;at java.lang.Thread.run(Thread.java:745)&lt;BR /&gt;...&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;===========&lt;/SPAN&gt;&lt;/P&gt;</description>
    <pubDate>Thu, 02 Mar 2017 22:13:55 GMT</pubDate>
    <dc:creator>IgorYakushin</dc:creator>
    <dc:date>2017-03-02T22:13:55Z</dc:date>
  </channel>
</rss>

