Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Custom Spark 2.2 on HDP 2.6 remote execution fails with HDFS connection refused message on port 8020

Custom Spark 2.2 on HDP 2.6 remote execution fails with HDFS connection refused message on port 8020

New Contributor

I have installed Spark 2.2 on a HDP 2.6 cluster. I am having issues submitting a spark-submit remote job from a machine outside from the Spark Cluster running on YARN:

Exception in thread "main" java.net.ConnectionException: Call from remote.dev.local/192.168.10.65 to target.dev.local:8020 failed on connection exception: java.net.ConnectionException: Connection Refused 

In my core-site.xml:

<property>
      <name>fs.defaultFS</name>
      <value>hdfs://target.dev.local:8020</value>
    <property> 

Also at my hdfs-site.xml in the cluster I have disabled permissions checking for HDFS:

<property> <name>dfs.permissions.enabled</name> <value>false</value> <property>

Also, when I telnet from the machine outside the cluster:

telnet target.dev.local 8020 

I am getting

telnet: connect to address 192.168.10.186: Connection Refused 

But, when I telnet target.dev.local 9000

it says Connected.

Also when I `ping target.dev.local` it works.

My spark-submit script from the remote machine is:

    export HADOOP_CONF_DIR=/<path_to_conf_dir_copied_from_cluster>/ 
    spark-submit --class org.apache.spark.examples.SparkPi \
    --master yarn \
    --deploy-mode cluster \ 
    --driver-memory 5g \    
    --executor-memory 50g \ 
    --executor-cores 5 \  
    --queue default \ 
     <path to jar>.jar \
    10 

Note that submitting this same job from the Master Node works fine.

What am I missing here?

Don't have an account?
Coming from Hortonworks? Activate your account here