- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Insert overwrite query failing with Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
- Labels:
-
Apache Hadoop
-
Apache Hive
Created ‎01-11-2016 05:05 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am a running a insert overwrite query as below.
Insert overwrite directory '/org/data/tmp/webapptempspace/UC3/log' select a.* from a join b on ucase(a.name)=ucase(b.name);
It works fine when one of the table has smaller dataset, but when both tables have huge data, it throws the below error.
Failed with exception Unable to move source /org/data/tmp/webapptempspace/UC3/log/.hive-staging_hive_2016-01-11_04-31-06_067_6297667876520454770-1/-ext-10000 to destination
Created ‎01-12-2016 03:40 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@pooja khandelwal Please check the hive log details and see if you have permission to write.
Created ‎01-12-2016 04:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, I have permission to write. I am able to run the query for a smaller dataset, but not for a larger dataset.
Created ‎02-29-2016 04:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I met the same problem. @pooja khandelwal has this been resolved? Can you provide your solution or accept the best answer?
Created ‎05-06-2020 03:24 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This could be permission issue. you can see the hive server2 log for the error. Log will be in /var/log/hive on the node to which you connect the hive
Created ‎01-12-2016 04:58 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My table 'a' has text format and 'b' has orc format. When I keep both as text, it is working fine.
Created ‎01-14-2016 04:56 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is there any more information in the logs? (If you're using HiveServer2, the you can look at those logs, or your client logs if you are using the Hive client directly.) The error is caused by a file system error where it's unable to move the file. The file system should log an error message describing why it failed.
Created ‎01-14-2016 05:23 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What MoveTask does is moving some files in /tmp volume to /user volume. When the user running doesn't have the right permissions it does not allow moving files between volumes and throws this exception.
Possible workarounds-
-Check /user and /tmp have the full permission
-Check if the below guys are set to true
hive.metastore.client.setugi=true and
hive.metastore.server.setugi=true. These parameters instruct hive execute jobs under current shell user
if not then try executing it as root.
Created ‎01-14-2016 05:50 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Check this MoveTask implemetation
Created ‎02-03-2016 01:51 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@pooja khandelwal has this been resolved? Can you provide your solution or accept the best answer?
