can weuse only SparkSQL API to solve all tasks? ... View more

Try this to access local file in Yarn Mode: import subprocess rdata =subprocess.check_output("cat /home/xmo3l1n2/pyspark/data/yahoo_stocks.csv", shell=True) pdata=sc.parallelize(rdata.split('\n')) ... View more