Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Multiple files to windows share using PutFile failing when files written rapidly

Highlighted

Multiple files to windows share using PutFile failing when files written rapidly

New Contributor

I have mounted a windows share (using fstab) on a Linux VM running NiFi.
I have setup the windows share permissions correctly as I am able to write/read to/from this share from the VM.

I want to write to the mount point of this share, flowfiles that are rapidly generated in NiFi using the PutFile processor.
Note: I am using the uuid of each flowfile for the file's filename.

The process succeeds for flowfiles generated at a slow rate (2s interval) but fails at a rapid rate (200ms interval) with the error message 'Can't rename file'.

I have noticed that the filename whilst the file is been written ( dot filename state ) changes to the next incoming flowfile filename prior to the completion of the current file write. Hence, creating a failure situation.

If I rapidly write files to a VM local directory using PutFile, all are successfully written!

Has anyone any suggestion on possibly configuring the PutFile processor (eg increasing the Concurrent Tasks?) to get this working? Do I need to do something on the windows end?