Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

How to save results from dataframe into a separate hive table

avatar

I have got the following:

val df = sqlContext.sql("SELECT * from table1")
var tempResult = df.filter(df("field1") > 10)

I have also already created another table - table2 - with the same structure as table1.

How can I save/insert the result of tempResult into table2?

1 ACCEPTED SOLUTION

avatar

OK - I get this working now. If anyone interested, here you are:

val df = sqlContext.sql("SELECT * from table1")
val tempResult = df.filter(df("field1") > 10)
tempResult.write.mode("overwrite").saveAsTable("default.new_table")
val df1 = sqlContext.sql("SELECT * from default.new_table")
df1.show()

NOTE: the "new_table" table can but does not need to exist before writing to it

View solution in original post

1 REPLY 1

avatar

OK - I get this working now. If anyone interested, here you are:

val df = sqlContext.sql("SELECT * from table1")
val tempResult = df.filter(df("field1") > 10)
tempResult.write.mode("overwrite").saveAsTable("default.new_table")
val df1 = sqlContext.sql("SELECT * from default.new_table")
df1.show()

NOTE: the "new_table" table can but does not need to exist before writing to it