Reply
Highlighted
Explorer
Posts: 12
Registered: ‎04-13-2018
Accepted Solution

problem in moving the input text file from file system to hdfs dfs file system with following comman

1) [cloudera@localhost ~]$ sudo -u hdfs hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: `/home/cloudera/ipf.txt': No such file or directory
2) [cloudera@localhost ~]$ hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

3) [cloudera@localhost ~]$ sudo -u cloudera hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

Expert Contributor
Posts: 338
Registered: ‎01-25-2017

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

[ Edited ]

1) [cloudera@localhost ~]$ sudo -u hdfs hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: `/home/cloudera/ipf.txt': No such file or directory

 

The file /home/cloudera/ipf.txt doesn't exist in you local host, you can check by ll /home/cloudera/

 

Below you are not using the sudo -u hdfs as you used in the above command.

 

** you faced the same issue in another post.

 

Please use sudo -u hdfs hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
2) [cloudera@localhost ~]$ hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

3) [cloudera@localhost ~]$ sudo -u cloudera hdfs dfs -put /home/cloudera/ipf.txt /inputnew/
put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

Explorer
Posts: 12
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

actually the txt file is already there in the folder and i am issuing the correct command but still i getting error.can u suggest me a solution

 

Expert Contributor
Posts: 338
Registered: ‎01-25-2017

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

PLease send me the output of the ll command
Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

1) User hdfs does not have access to the /home/cloudera directory

2) and 3) is actually the same because in both cases you try to upload the file as user cloudera.

 

You have two options:

1) grant read permissions to hdfs user in /home/cloudera and all sub-contents (directory access require also execute permission)

2) grant write permissions in "/inputnew/" directory in HDFS , to "cloudera" user.

example: sudo -u hdfs hdfs dfs -chown cloudera /inputnew

 

There are multiple ways to grant permissions (e.g. using ACLs), but keep it simple.

Explorer
Posts: 12
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

put: Permission denied: user=cloudera, access=WRITE, inode="/inputnew":hdfs:supergroup:drwxr-xr-x

Explorer
Posts: 12
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

how to give permission to cloudera
Expert Contributor
Posts: 133
Registered: ‎01-08-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

Use the example I wrote above. It will change the owner of /inputnew directory in hdfs to "cloudera"

sudo -u hdfs hdfs dfs -chown cloudera /inputnew
Explorer
Posts: 12
Registered: ‎04-13-2018

Re: problem in moving the input text file from file system to hdfs dfs file system with following co

thank u,its working
Explorer
Posts: 12
Registered: ‎04-13-2018

Excution of the following command gives warning and not executed

hadoop jar /home/cloudera/WordCount.jar WordCount /inputnew/inputfile.txt /outputnew

 

error message is:

[cloudera@localhost ~]$ hadoop jar /home/cloudera/WordCount.jar WordCount /inputnew/inputfile.txt /outputnew
18/04/18 07:57:24 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
18/04/18 07:57:25 INFO input.FileInputFormat: Total input paths to process : 1
18/04/18 07:57:25 INFO mapred.JobClient: Running job: job_201804180742_0002
18/04/18 07:57:26 INFO mapred.JobClient: map 0% reduce 0%
18/04/18 07:57:36 INFO mapred.JobClient: Task Id : attempt_201804180742_0002_m_000002_0, Status : FAILED
org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:204)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4705)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4687)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4661)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3032)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:2996)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2977)

 

Announcements