Reply
New Contributor
Posts: 10
Registered: ‎04-13-2018
Accepted Solution

problem in moving the input text file from file system to hdfs dfs file system with following comman

1) [cloudera@localhost ~]$ sudo -u hdfs hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: `/home/cloudera/ipf.txt': No such file or directory
2) [cloudera@localhost ~]$ hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

3) [cloudera@localhost ~]$ sudo -u cloudera hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

Expert Contributor
Posts: 304
Registered: ‎01-25-2017

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

[ Edited ]

1) [cloudera@localhost ~]$ sudo -u hdfs hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: `/home/cloudera/ipf.txt': No such file or directory

 

The file /home/cloudera/ipf.txt doesn't exist in you local host, you can check by ll /home/cloudera/

 

Below you are not using the sudo -u hdfs as you used in the above command.

 

** you faced the same issue in another post.

 

Please use sudo -u hdfs hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
2) [cloudera@localhost ~]$ hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

3) [cloudera@localhost ~]$ sudo -u cloudera hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

New Contributor
Posts: 10
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

actually the txt file is already there in the folder and i am issuing the correct command but still i getting error.can u suggest me a solution

 

Expert Contributor
Posts: 304
Registered: ‎01-25-2017

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

PLease send me the output of the ll command
Highlighted
Expert Contributor
Posts: 80
Registered: ‎01-08-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

1) User hdfs does not have access to the /home/cloudera directory

2) and 3) is actually the same because in both cases you try to upload the file as user cloudera.

 

You have two options:

1) grant read permissions to hdfs user in /home/cloudera and all sub-contents (directory access require also execute permission)

2) grant write permissions in "/inputnew/" directory in HDFS , to "cloudera" user.

example: sudo -u hdfs hdfs dfs -chown cloudera /inputnew

 

There are multiple ways to grant permissions (e.g. using ACLs), but keep it simple.

New Contributor
Posts: 10
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

New Contributor
Posts: 10
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

how to give permission to cloudera
Expert Contributor
Posts: 80
Registered: ‎01-08-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

Use the example I wrote above. It will change the owner of /inputnew directory in hdfs to "cloudera"

sudo -u hdfs hdfs dfs -chown cloudera /inputnew
New Contributor
Posts: 10
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

thank u,its working
New Contributor
Posts: 10
Registered: ‎04-13-2018

Excution of the following command gives warning and not executed

hadoop jar /home/cloudera/WordCount.jar WordCount /inputnew/inputfile.txt /outputnew

 

error message is:

[cloudera@localhost ~]$ hadoop jar /home/cloudera/WordCount.jar WordCount /inputnew/inputfile.txt /outputnew
18/04/18 07:57:24 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
18/04/18 07:57:25 INFO input.FileInputFormat: Total input paths to process : 1
18/04/18 07:57:25 INFO mapred.JobClient: Running job: job_201804180742_0002
18/04/18 07:57:26 INFO mapred.JobClient: map 0% reduce 0%
18/04/18 07:57:36 INFO mapred.JobClient: Task Id : attempt_201804180742_0002_m_000002_0, Status : FAILED
org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:204)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4705)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4687)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4661)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3032)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:2996)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2977)

 

Announcements