I use Hortonworks Sandbox HDP 2.6.5 and putty to use Linux shell. My OS is window10.
I put some JSON file on HDFS and I want to open this file with pyspark.
I use below python file on linux, typing "spark-submit example.py" on shell
But I got this error message
"Call From sandbox-hdp.hortonworks.com/172.18.0.2 to localhost:8020 failed on connection exception"
I don't know what's wrong, can you help me please?
You need to give the complete path of the filesystem. You can just change it like below
jsonData = spark.read.json("/user/maria_dev/example.json")
Spark will figure it out from fs.defaultFS value. If you still want to pass the complete path, then you can try with the IP address of the namenode instead of localhost.