Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Solved
Go to solution
How to save results from dataframe into a separate hive table
Labels:
- Labels:
-
Apache Spark
Guru
Created 02-20-2017 08:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have got the following:
val df = sqlContext.sql("SELECT * from table1") var tempResult = df.filter(df("field1") > 10)
I have also already created another table - table2 - with the same structure as table1.
How can I save/insert the result of tempResult into table2?
1 ACCEPTED SOLUTION
Guru
Created 02-20-2017 02:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK - I get this working now. If anyone interested, here you are:
val df = sqlContext.sql("SELECT * from table1") val tempResult = df.filter(df("field1") > 10) tempResult.write.mode("overwrite").saveAsTable("default.new_table") val df1 = sqlContext.sql("SELECT * from default.new_table") df1.show()
NOTE: the "new_table" table can but does not need to exist before writing to it
1 REPLY 1
Guru
Created 02-20-2017 02:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK - I get this working now. If anyone interested, here you are:
val df = sqlContext.sql("SELECT * from table1") val tempResult = df.filter(df("field1") > 10) tempResult.write.mode("overwrite").saveAsTable("default.new_table") val df1 = sqlContext.sql("SELECT * from default.new_table") df1.show()
NOTE: the "new_table" table can but does not need to exist before writing to it
