Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

[#1]

Problem loading data to table in this tutorial

http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/

This command

<code>LOAD DATA INPATH '/user/admin/Batting.csv' OVERWRITE INTO TABLE temp_batting;

produces error

H110 Unable to submit statement. Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [admin] does not have [READ] privilege on [hdfs://sandbox.hortonworks.com:8020/user/admin/elecMonthly_Orc] [ERROR_STATUS]

I created both user/admin and temp/admin folders. I used hdfs superuser to make admin owner of file, folder, and even parent folder. I gave full permissions in HDFS. and this is clearly shown in Ambari. Error persists.

Can anyone help? Thanks

[#2]

@Aidan Condron

Can you please take a screenshot of the output of

hdfs dfs -ls /user/admin/elecMonthly_Orc

[#3]

@Aidan Condron what user are you running the Hive Statement as?

[#4]

Thanks Andrew. Think it was discongruity between Hive account running statement and Admin owing file. Thanks for this and sorry for delay in reply.

[#5]

@Aidan Condron Is it resolved?

[#6]

@Aidan Condron accept best answer below

[#7]

Are you using the latest sandbox? Try the Ambari Hive view, it has the Upload File action specifically for csv files.

[#8]

Thanks for answering so quickly!

[#9]

@Aidan Condron what about the directory about it. What's the output of:

hdfs dfs -ls /user/admin

It looks like the files are owned by the user 'Spark'. Which user is the running the Hive Statement?

[#10]

admin is running statement as per tutorial

I thought I did hdfs chown on files. Shown in Ambari as owned by admin

[#11]

@Aidan Condron

Is temp_batting an external or internal table? My assumption is it's an internal table and thus when you load data in path, it's trying to move the data in /apps/warehouse/database(default)/temp_batting and the admin user doesn't have permissions to move the file.

Can you please try running:

hdfs dfs -chmod -R 777 /user/admin/elecMonthly_Orc

and then trying to run your load data inpath command?

[#12]

@Aidan Condron are you still having issues with this? Can you accept best answer or provide your own solution?

[#13 · Accepted solution]

It was because when I thought I was creating an elecMonthly_Orc file, actually I created an elecMonthly_Orc folder with several files three files: _SUCCESS, part-r-r-00000-1a0c14e3-0dd0-42db-abc7-7f655a02f634.orc ... and another similar orc files. The files within the elecMonthly_Orc directory were owned by Hive, and that's why the permissions error.

Resolved by using command line as superuser hdfs:

hadoop fs -chown admin:admin /user/admin/elecMonthly_Orc/*.*

Now I just have to figure out how to recombine Orc files in Hive!

[#14]

@Aidan Condron can you convert your response to answer and we can mark it as best answer?

[#15]

Hey guys. The tutorial mentioned above has been updated and is also compatible with the latest Sandbox HDP 2.4. It addresses the issue of permissions. Here is the link: http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/

When you a chance, can you go through the tutorial on our new Sandbox?