Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 11-19-2015 03:03 AM
I'm currently using Spark 1.4 and I'm loading some data into a DataFrame using jdbc:
val jdbcDF = sqlContext.load("jdbc", options)
How can I save the jdbcDF DataFrame to a Hive table using the ORC file format?
Created 11-19-2015 03:07 AM
df.write.format("orc") will get you there.
See: http://hortonworks.com/blog/bringing-orc-support-into-apache-spark/ or http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_spark-guide/content/ch_orc-spark.html
Created 03-25-2016 03:45 PM
@Brandon Wilson I tried your suggestion it creates the hive table but I get this error:
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to alter table.
and it does not load data into my table. do you have any idea how to solve this?
Created 12-17-2015 02:21 AM
Divya,
What is the user account when DF is used to create the external Hive table?
What is the user account when you try to see the table in Hive (& did you use HiveCli or Hive/Beeline or some ODBC tool?)
Created 12-18-2015 06:26 AM
@vshukla I am logging in as hdfs user on HDP 2.3.2 sandbox
and using the same account to see tables in hive.Yes , I am using hive CLI and even browsed HDFS files through Ambari .Couldnt see any tables created.