Hive error on HDP6

[#1]

I'm keep getting permission error while loading data to Hive tables and/or querying tables. I granted full(r+w+x) permission to appl/hadoop/yarn/local/usercache in all of my cluster nodes. I'm still getting errors. I have 3 nodes cluster (HDP6).

Application application_1507209932823_0008 failed 2 times due to AM Container for appattempt_1507209932823_0008_000002 exited with exitCode: -1000 For more detailed output, check the application tracking page: http://mymaster.fisdev.local:8088/cluster/app/application_1507209932823_0008 Then click on links to logs of each attempt. Diagnostics: Could not set permissions for local dir /appl/hadoop/yarn/local/usercache Failing this attempt. Failing the application.

[#2]

I noticed /appl/hadoop/yarn on different nodes had different owner/grp. I made them all to yarn:hadoop

before:

[yarn@vlmazgrpmaster ~]$ ls -ld /appl/hadoop/yarn drwxrwxrwx 4 root hadoop 30 Aug 24 14:10 /appl/hadoop/yarn [e1009156@vlmazgrpdata1 ~]$ ls -ld /appl/hadoop/yarn drwxrwxrwx 4 root root 30 Aug 24 14:36 /appl/hadoop/yarn [e1009156@vlmazgrpdata2 ~]$ ls -ld /appl/hadoop/yarn drwxrwxrwx 4 root hadoop 30 Aug 24 14:35 /appl/hadoop/yarn

now:

drwxrwxrwx 4 yarn hadoop 30 Aug 24 14:35 /appl/hadoop/yarn

[#3 · Accepted solution]

@Winnie Philip

After fixing the user:group .. is it working Or still failing

drwxrwxrwx 4 yarn hadoop 30 Aug 24 14:35 /appl/hadoop/yarn

.

[#4]

It worked 1/3 test. it works sometimes, but not always:

I checked all nodemanager directories, they all looked same owner:grp:

ls -ld /hadoop/yarn/local drwxr-xr-x 6 yarn hadoop 78 Oct 6 12:56 /hadoop/yarn/local ls -ld /appl/hadoop/yarn/local drwxrwxrwx 5 yarn hadoop 57 Oct 6 12:56 /appl/hadoop/yarn/local ls -ld /usr/db/hadoop/yarn/local drwxr-xr-x 5 yarn hadoop 57 Oct 6 12:56 /usr/db/hadoop/yarn/local ls -ld /usr/local/opt/hadoop/yarn/local drwxr-xr-x 5 yarn hadoop 57 Oct 6 12:58 /usr/local/opt/hadoop/yarn/local

[#5]

I ran the command with -R, looks like fixed it, I will watch for errors little bit, I restarted YARN also, this directory tree had mixed owner:grp.

chown yarn:hadoop -R /appl/hadoop/yarn

[#6]

@Winnie Philip

Great!!! to see your issue is fixed. It will be also great if you can mark this HCC thread as Answered by clicking on the "Accept" Button. That way other HCC users can quickly find the solution when they encounter the same issue.

[#7]

Also, I went ahead and ran commands for all other nodemanager yarn directory chain on all nodes:

• sudo chown -R yarn:hadoop -R /appl/hadoop/yarn
• sudo chown -R yarn:hadoop /hadoop/yarn/
• sudo chown -R yarn:hadoop /usr/db/hadoop/yarn/
• sudo chown -R yarn:hadoop /usr/local/opt/hadoop/yarn/

Thank you very much for all help from everyone! I really appreciate!!