About barlow

VidyaSargur · ‎06-15-2020

@shubh As this is an older post that has been marked solved in 2018. You would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment that could aid others in providing a more accurate answer to your question.

barlow · ‎10-21-2018

Hi Shu, thanks for responding. The solution you provided appears a little difficult for something that I thought would be relatively simple. I will try your solution and let you know how I get on. In the meantime, have you seen the solution provided here: https://forums.databricks.com/questions/2848/how-do-i-create-a-single-csv-file-from-multiple-pa.html?childToView=12091

barlow · ‎10-19-2018

Hi guys, I'm sorry if the question seems a little confusing. Basically, I would just like to be able to save to a single file and the file to be overwritten each time it is saved. Thanks

Yuexin Zhang · ‎08-14-2018

As the error message states, the object, either a DataFrame or List does not have the saveAsTextFile() method. result.write.save() or result.toJavaRDD.saveAsTextFile() shoud do the work, or you can refer to DataFrame or RDD api: https://spark.apache.org/docs/2.1.0/api/scala/index.html#org.apache.spark.sql.DataFrameWriter https://spark.apache.org/docs/2.1.0/api/scala/index.html#org.apache.spark.rdd.RDD

sandyy006 · ‎08-13-2018

As this is duplicate, Please delete/close this question.

sandyy006 · ‎08-13-2018

@Carlton Patterson Looks like you have accepted another comment. I've made this reply as comment and this should be the correct one to accept as it helped in resolving your issue. 🙂

SQLShaw · ‎02-04-2018

@Carlton Patterson I may be a little biased, but this book might help. https://www.amazon.com/Practical-Hive-Hadoops-Warehouse-System/dp/1484202724/ref=sr_1_1?ie=UTF8&qid=1517772085&sr=8-1&keywords=practical+hive

JordanMoore · ‎02-02-2018

I assume this returns a limited result set, though, for large tables?

nramanaiah · ‎01-31-2018

@Carlton Patterson pls try the below create statement for acorn_category_frequency table creation Create table acorn_category_frequency ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/sampleout/' as select acorn_category, count(*) as acorn_categorycount from geography group by acorn_category<br>

smnpatil1591 · ‎06-14-2018

This could be because you are parsing actual data in the place of header,supposing your first row has header and second row onwards has data. Hence it can't parse data(int, string) as header(string). So try changing it to ("skip.header.line.count"="1"); Hope this helps.

Online	Offline
Last Visited	‎08-14-2018 02:05 PM

Date Registered	‎08-05-2018 02:01 AM
Last Visited	‎08-14-2018 02:05 PM
Total Messages Posted	73

Re: How to save all the output of pyspark sql quer...

Re: Unable to Create a single file with PySpark qu...

Re: How to overwrite a file with pyspark

Re: Pyspark issue AttributeError: 'DataFrame' obje...

Re: How to concatenate date to file in pyspark

Re: How to concatenate a date to a filename in pys...

Re: Hive Query Language Book Recommendations

Re: How to store Query Results to Local Drive

Re: Unable to Store Results from Hive Query

Re: ParseException missing EOF error message with ...