question Re: MR's Jobtracker can't be started due to permission issue in Archives of Support Questions (Read Only)

MR's Jobtracker can't be started due to permission issue

icecreamlc — Fri, 16 Sep 2022 09:06:58 GMT

I were tying to follow the example (http://blog.cloudera.com/blog/2012/09/automating-your-cluster-with-cloudera-manager-api/) to create and start MR service.

The task tracker can be started, the job tracker can't be started due to the following file permission reason.

My HDFS serivce use 'simple' permission. who can tell me if i need further configuration to grant user=mapred to write the folder Using CM API?? Thanks a lo!

2014-09-05 12:04:44,161 WARN org.apache.hadoop.mapred.JobTracker: Failed to operate on mapred.system.dir (hdfs://dhcp-corp-233.sc-cig-eng.tst:8020/tmp/mapred/system) because of permissions.
2014-09-05 12:04:44,162 WARN org.apache.hadoop.mapred.JobTracker: This directory should be owned by the user 'mapred (auth:SIMPLE)'
2014-09-05 12:04:44,163 WARN org.apache.hadoop.mapred.JobTracker: Bailing out ...
org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

Re: MR's Jobtracker can't be started due to permission issue

Darren — Fri, 05 Sep 2014 20:04:12 GMT

Did you do the step to create the HDFS /tmp directory? This is described via command line in the blog post you linked, but there is also an API command to do this.

Those instructions are fairly dated. You should manually set up a cluster, look at all the steps performed by First Run when initially setting up the cluster, and make sure you do all of those steps.

Re: MR's Jobtracker can't be started due to permission issue

icecreamlc — Fri, 05 Sep 2014 20:39:31 GMT

do you know how to use API to run the command?

I was trying to use API to automate cdh installation. what's the best way to do that? thanks a million!

Re: MR's Jobtracker can't be started due to permission issue

Darren — Fri, 05 Sep 2014 20:41:31 GMT

Use the API command to create the HDFS temp dir. Were you not able to find it in your version of the API?

Re: MR's Jobtracker can't be started due to permission issue

icecreamlc — Fri, 05 Sep 2014 20:50:10 GMT

which function for this API? sorry, i couldn't find the right one.

Re: MR's Jobtracker can't be started due to permission issue

icecreamlc — Fri, 05 Sep 2014 20:51:37 GMT

probably this is the answer. i will check this...

create_hdfs_tmp

Re: MR's Jobtracker can't be started due to permission issue

icecreamlc — Fri, 05 Sep 2014 21:01:59 GMT

I tested and worked! Thanks a lot!

What i used is the function below:

reate_hdfs_tmp(self)

source code

Create the /tmp directory in HDFS with appropriate ownership and permissions.

Returns: Reference to the submitted command

Since: API v2