- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Nifi: file not picked from input directory after first ime
- Labels:
-
Apache NiFi
Created 09-14-2023 04:21 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a nifi flow, working fine. It picks file from a source directory, processes it, and file is deleted from source directory once picked up.
This all works fine first time. But later on, if I copy another file in source directory, that file is not picked up. I even stop all processors, clean any queue if not empty, and restart processors, but still file is not picked from input directory.
What causes this behaviour?
As a result, I have to stop-restart nifi, making it a bit cumbersome.
PS: Based on some google suggestions, I tried to delete flowfile_repository, but it doesn't allow so with message 'Device or resource busy'.
Created on 09-14-2023 04:41 AM - edited 09-14-2023 04:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Got the answer. First processor has listing strategy set to 'Tracking Timestamps', and copied file has older timestamp than the latest tracked one.
Created on 09-14-2023 04:41 AM - edited 09-14-2023 04:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Got the answer. First processor has listing strategy set to 'Tracking Timestamps', and copied file has older timestamp than the latest tracked one.
Created 09-14-2023 09:55 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@manishg
Copying a file should have resulted in a new timestamp making it newer than the last file you previously listed. However, moving a file typically does not change timestamps on the file. My guess here is you moved a file into the directory instead of copy?
Matt
Created 09-14-2023 07:24 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I used docker copy command.
Created 09-14-2023 08:17 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What processors are you using? List or Get files?
Created 09-14-2023 07:25 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Its ListFile.
