Support Questions
Find answers, ask questions, and share your expertise

loading a text file to hadoop fs for later re-reading and conversion to csv

Highlighted

loading a text file to hadoop fs for later re-reading and conversion to csv

New Contributor

Hi, i have a text file that i output my results to and save it on my local edge node, running pyspark. i tried reading it, but since it's not on the cluster, spark can't get to it due to file path not found. 

how do i get the file on to hdfs as i would with spark using '-put' for later re-reading and conversion to csv, for automated data collection and conversion rather than having to manually dowload the file and do te conversion.

 

thank you in advance.