question Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables in Archives of Support Questions (Read Only)

Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

acondron — Tue, 05 Jan 2016 22:21:25 GMT

Problem loading data to table in this tutorial

http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/

This command

<code>LOAD DATA INPATH '/user/admin/Batting.csv' OVERWRITE INTO TABLE temp_batting;

produces error

H110 Unable to submit statement. Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [admin] does not have [READ] privilege on [hdfs://sandbox.hortonworks.com:8020/user/admin/elecMonthly_Orc] [ERROR_STATUS]

I created both user/admin and temp/admin folders. I used hdfs superuser to make admin owner of file, folder, and even parent folder. I gave full permissions in HDFS. and this is clearly shown in Ambari. Error persists.

Can anyone help? Thanks

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

awatson — Tue, 05 Jan 2016 22:26:01 GMT

@Aidan Condron

Can you please take a screenshot of the output of

hdfs dfs -ls /user/admin/elecMonthly_Orc

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

awatson — Tue, 05 Jan 2016 22:29:06 GMT

@Aidan Condron what user are you running the Hive Statement as?

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

andrewg — Tue, 05 Jan 2016 22:33:52 GMT

Are you using the latest sandbox? Try the Ambari Hive view, it has the Upload File action specifically for csv files.

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

acondron — Mon, 19 Aug 2019 12:20:36 GMT

Thanks for answering so quickly!

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

awatson — Tue, 05 Jan 2016 22:38:54 GMT

@Aidan Condron what about the directory about it. What's the output of:

hdfs dfs -ls /user/admin

It looks like the files are owned by the user 'Spark'. Which user is the running the Hive Statement?

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

acondron — Mon, 19 Aug 2019 12:20:28 GMT

admin is running statement as per tutorial

I thought I did hdfs chown on files. Shown in Ambari as owned by admin

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

awatson — Tue, 05 Jan 2016 23:24:57 GMT

@Aidan Condron

Is temp_batting an external or internal table? My assumption is it's an internal table and thus when you load data in path, it's trying to move the data in /apps/warehouse/database(default)/temp_batting and the admin user doesn't have permissions to move the file.

Can you please try running:

hdfs dfs -chmod -R 777 /user/admin/elecMonthly_Orc

and then trying to run your load data inpath command?

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

aervits — Wed, 03 Feb 2016 03:28:59 GMT

@Aidan Condron are you still having issues with this? Can you accept best answer or provide your own solution?

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

acondron — Wed, 03 Feb 2016 19:08:09 GMT

Thanks Andrew. Think it was discongruity between Hive account running statement and Admin owing file. Thanks for this and sorry for delay in reply.

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

nsabharwal — Wed, 03 Feb 2016 19:10:14 GMT

@Aidan Condron Is it resolved?

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

aervits — Wed, 03 Feb 2016 19:48:20 GMT

@Aidan Condron accept best answer below

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

acondron — Thu, 04 Feb 2016 00:17:55 GMT

It was because when I thought I was creating an elecMonthly_Orc file, actually I created an elecMonthly_Orc folder with several files three files: _SUCCESS, part-r-r-00000-1a0c14e3-0dd0-42db-abc7-7f655a02f634.orc ... and another similar orc files. The files within the elecMonthly_Orc directory were owned by Hive, and that's why the permissions error.

Resolved by using command line as superuser hdfs:

hadoop fs -chown admin:admin /user/admin/elecMonthly_Orc/*.*

Now I just have to figure out how to recombine Orc files in Hive!

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

aervits — Thu, 04 Feb 2016 00:19:42 GMT

@Aidan Condron can you convert your response to answer and we can mark it as best answer?

Re: Error on Tutorial "How to Process Data with Apache Hive" - Cannot load data into Hive tables

jmedel — Tue, 08 Mar 2016 10:30:43 GMT

Hey guys. The tutorial mentioned above has been updated and is also compatible with the latest Sandbox HDP 2.4. It addresses the issue of permissions. Here is the link: http://hortonworks.com/hadoop-tutorial/how-to-process-data-with-apache-hive/

When you a chance, can you go through the tutorial on our new Sandbox?