@Madhav_VD  What about the "Relationship" tab? My guess here is that you have checked the "retry" box on the success relationship of the putSQL processor.  If that is the case, unchecking "retry" on the success relationship should resolve your FlowFile penalization issue: When this processor is running, does it produce any bulletins or exceptions in the log output?  If it is producing bulletins, warn, or error logs, it is likely failing to write to your SQL DB.  The FlowFIle would then be routed to retry or failure relationship depending on exception where "retry" if checked would be applied based on retry property configurations.  If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@joyjlee  The File-User-Group-Provider is responsible for: 1. Creating the users.xml using the configured provider properties ONLY if the users.xml file does not already exist. 2. Loading the users from the users.xml in to heap memory on startup when the users.xml already exists. The File-Access-Policy-Provider is responsible for: 1. Generating the authorizations.xml file ONLY if it does not already exist using the configured provider properties.  This provider will not work if users are not already loaded by some user group provider (multiple options exist for loading users: file-user-group-provider, ldap-user-group-provider, shell-user-group-provider, etc.) 2. Loading the user's authorizations into memory if the authorizations.xml file already exists. So by creating your own custom users.xml and authorizations.xml files, these providers do nothing beyond loading what is already in those files into memory.  They will not append to or modify them. NiFI does not support wildcard policies either. The UUID generated by NiFi for a User Identity string will always be the same. So I am assuming you captured all those UUID for the users.xml you are manually creating. Also NiFi only generates the flow.json.gz (replacing flow.xml.gz) if it does not already exist. While upon creation initially the UUID for the root process group will be random, you could certainly load a pre-built flow.json.gz with your deployment that only has that root PG already in it.   That would allow your pre-built users.xml and authorizations.xml to load with policies you defined. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@enam  Looks like you have a bad file filter regex in your listSFTP processor configuration. .*file.*\.xls Above looks for any character for 1 or more characters until is finds the last occurrence of string "file" followed by any character for as many characters until last occurrence of string ".xls".  However, all your filenames start with "file" and have no characters before it. Try modifying your file filter regex by removing the ".*" before "file": file.*\.xls  Right click on processor, select "view state", and then "clear state".  Then start the listSFTP processor again to see if generated NiFi FlowFiles for each file on your SFTP server. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more