@manishg  The FetchFile processor fetches the content of the target file and inserts its content into the FlowFile that triggered the Fetch.  That FlowFiles is then passed on to the next processor.  So whatever is set as the "filename" attribute on the FlowFile will remain the filename even after fetching the content.  Without the specific on the rest of your dataflows configuration, it is hard to provide any additional input on your issue. My guess here is that the putSFTP processor is writing the content of the fetched file to the target SFTP server location; however, its filename is not that of the fetched file.  If this is the case, you have a datafow design issue and need to check your configurations to make sure the filename attribute is being set accordingly. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Actually there is a property called "Move Destination Directory" : As the description says on this properties: "this property is ignored unless the Completion Strategy is set to Move".  The same rule applies on the Move Conflict Strategy as its only considered when the Completion Strategy is set to Move . If your Completion Strategy is set to None or Delete then both properties will be ignored. ... View more

Ok, I think I know what you mean now.  Basically the GetFile is designed not to take any incoming connection as you can see in the processor documentation: The reason  such restriction applies depend on how the processor works. The GetFile is designed as a trigger processor and it executes every time a file is placed in the target path, so providing an incoming relationship (wither through routeOnAttribute or any other processor) will break this rule because that means the GetFile will only work and Get the file when upper stream processor triggers that via sending flowfile which is not what this processor is designed to do. What you need is FetchFile processor given that you know the file you are looking for because this processor takes the full file path and not just a directory.   ... View more

@manishg  Same about of flowFiles per second processing after switching to the Volatile repositories? Perhaps having FlowFile and provenance repositories in memory allows for faster processing of FlowFIles resulting in more read and writes to the content_repository which contains the actual content of each FlowFile. If your NiFi should crash or restart you will lose everything in your volatile repositories.  The FlowFile repository holds all the FlowFile metadata for the FlowFiles currently being processed through your dataflows.   This means data loss in such events. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

InvokeScriptedProcessor is the closest you'll get to a native (NAR) NiFi processor from my experience. With it, you do NOT need to define all three relationships...if your code handles all possible problems correctly, you could just have "success" or other for your relationship. From what I've seen you do need to have at least one relationship if you're modifying the FlowFile or creating new ones and require session.transfer to send it to that relationship. You don't need to transfer the original...if you read the original FlowFile and create a new one or several new ones you can dispose of the original with session.remove(your_orginal_flow_file)  ... View more

Well the only advise I can give you is to write your processor and see what errors you have and come back with them. Nobody can write your processor if only you know your requirements. What I can suggest you though, is to have a look at the following examples, as they might assist you with what you are trying to achieve: https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-1/ta-p/248922 https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-2/ta-p/249018 https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-3/ta-p/249148   ... View more