Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Writing parquet file to hdfs for internal Hive table

avatar
Contributor

I'm trying to overwrite a parquet file in hdfs for a Hive table, but I noticed a bunch of individual parquet files with unique names like this, 

table_directory /part-00199-edf111ef-0f7a-43ab-9393-8fc620195d4c-c000.snappy.parquet

 

How do I add a parquet file to a Hive table directory that's composed of several parquet file?

 

 

2 ACCEPTED SOLUTIONS

avatar
Super Guru

@Wilber My suggestion would be to not over write existing files, but write to staging location as external table.  Then merge staging location data into final internal hive table with INSERT INTO final_table SELECT * FROM staging_table;

View solution in original post

avatar
Contributor

@stevenmatison  - perfect; works! Thank you.

View solution in original post

3 REPLIES 3

avatar
Super Guru

@Wilber My suggestion would be to not over write existing files, but write to staging location as external table.  Then merge staging location data into final internal hive table with INSERT INTO final_table SELECT * FROM staging_table;

avatar
Contributor

@stevenmatison  - perfect; works! Thank you.

avatar
Super Guru

Please accept the solution as answer.  Doin this helps complete the solution.