Support Questions

Find answers, ask questions, and share your expertise

How to save results from dataframe into a separate hive table

avatar

I have got the following:

val df = sqlContext.sql("SELECT * from table1")
var tempResult = df.filter(df("field1") > 10)

I have also already created another table - table2 - with the same structure as table1.

How can I save/insert the result of tempResult into table2?

1 ACCEPTED SOLUTION

avatar

OK - I get this working now. If anyone interested, here you are:

val df = sqlContext.sql("SELECT * from table1")
val tempResult = df.filter(df("field1") > 10)
tempResult.write.mode("overwrite").saveAsTable("default.new_table")
val df1 = sqlContext.sql("SELECT * from default.new_table")
df1.show()

NOTE: the "new_table" table can but does not need to exist before writing to it

View solution in original post

1 REPLY 1

avatar

OK - I get this working now. If anyone interested, here you are:

val df = sqlContext.sql("SELECT * from table1")
val tempResult = df.filter(df("field1") > 10)
tempResult.write.mode("overwrite").saveAsTable("default.new_table")
val df1 = sqlContext.sql("SELECT * from default.new_table")
df1.show()

NOTE: the "new_table" table can but does not need to exist before writing to it