- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to do multiple parallel inserts into hive external table
- Labels:
-
Apache Hive
Created ‎08-08-2018 05:31 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I could not find any way to do multiple inserts into hive table because of the naming "000000_0" that hive internally follows while creating external table. I was wondering if there is a way we can set this prefix filename so that we can do multiple parallel inserts.
Anyother way we can do parallel inserts into hive external table is also being looked for
Thanks in advance
Created ‎08-08-2018 01:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As indicated in an existing answer by @hduraiswamy there are some things you can do:
1. Give multiple insert commands in parallel, and they will automatically be executed sequentially
2. Writing multiple files to a directory and then creating a hive table on top of the folder, see the aforementioned answer
If this does not work for you, you can of course also work with a non-external hive table.
- Dennis Jaheruddin
If this answer helped, please mark it as 'solved' and/or if it is valuable for future readers please apply 'kudos'.
