Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to over write file with cp command through NFS Gateway into HDFS.

Highlighted

Unable to over write file with cp command through NFS Gateway into HDFS.

Contributor

I am trying to over write the file with cp command and it failed with below error.

cp: cannot create regular file "filename" : Invalid argument

i checked the error log and got the below error.

ERROR nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:setattr(436)) - Setting file size is not supported when setattr, fileId: 18463

Can some one Help me with it.

Thanks in Advance.

6 REPLIES 6

Re: Unable to over write file with cp command through NFS Gateway into HDFS.

Contributor

Hi, cp is working with the option "-f". But when i try with python it is failing with below error.

IOError: [Errno 22] invalid mode ('wb+') or filename: '/file_path/testWrite.txt'

Can some one help with it.

Thanks.

Re: Unable to over write file with cp command through NFS Gateway into HDFS.

New Contributor

Hi Paramesh,what is the python code you are using.

Re: Unable to over write file with cp command through NFS Gateway into HDFS.

Contributor

Hi Selavan,

First of all thanks for your reply. I used below code with the python write mode of w,w+ and wb+.

open_file = open("/hdfs_nfs/hdfs_Data/sampledata/testWrite.txt", 'wb+')
open_file.write("This is just a sample data")
open_file.close()

The file "testWrite.txt" is getting created in given path (HDFS Mount point) when I run this for first time.

I am getting below error, when I execute above code from second time onwards.

IOError: [Errno 22] invalid mode ('wb+') or filename: '/hdfs_nfs/hdfs_Data/sampledata/testWrite.txt'
Thanks.

Re: Unable to over write file with cp command through NFS Gateway into HDFS.

Guru

@Paramesh malla, Is testWrite.txt file present on HDFS while running test code second time ? If yes, please delete /hdfs_nfs/hdfs_data/sampledata/testWrite.txt and rerun.

HDFS only supports append, so if you intend to append data after file creation, use 'a' option.

with open(filename,'a')as f:
    f.write(text)

Re: Unable to over write file with cp command through NFS Gateway into HDFS.

Super Guru
@Paramesh malla

You need to overwrite file with python(like cp -f) in order to get it working.

Please refer below link:

https://stackoverflow.com/questions/2424000/read-and-overwrite-a-file-in-python

Hope this helps.

Re: Unable to over write file with cp command through NFS Gateway into HDFS.

Contributor

Hi Kuldeep,

Thanks for the reply. I tried the method that was mentioned on above given link. still it is not working.

I am getting below error.

Traceback (most recent call last):
  File "write_test.py", line 7, in <module>
    open_file.truncate()
IOError: [Errno 22] Invalid argument.