question Re: How to save results from dataframe into a separate hive table in Archives of Support Questions (Read Only)

How to save results from dataframe into a separate hive table

dkozlowski — Mon, 20 Feb 2017 16:52:57 GMT

I have got the following:

val df = sqlContext.sql("SELECT * from table1")
var tempResult = df.filter(df("field1") > 10)

I have also already created another table - table2 - with the same structure as table1.

How can I save/insert the result of tempResult into table2?

Re: How to save results from dataframe into a separate hive table

dkozlowski — Mon, 20 Feb 2017 22:19:34 GMT

OK - I get this working now. If anyone interested, here you are:

val df = sqlContext.sql("SELECT * from table1")
val tempResult = df.filter(df("field1") > 10)
tempResult.write.mode("overwrite").saveAsTable("default.new_table")
val df1 = sqlContext.sql("SELECT * from default.new_table")
df1.show()

NOTE: the "new_table" table can but does not need to exist before writing to it