@sha257  NiFi's ldap-user-group-provider does not support nested groups (NIFI-8035) Based on what you have shared, your user "W0YZ1" is not a direct memberOf group:   AG-X-SAMPLE-ADMIN   The user is a member of a bunch of other groups and I am guessing that one of more of these groups are a member of the above group. And since your user search filter is only going to return users that are a direct memberOf:    CN=AG-X-SAMPLE-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net   NiFi's ldap-user-group-provider is not going to return the sample user you shared above. Something you may want to try here: It is not required that both the user and group search properties are configured in order to get users and groups returned.  In your case I would suggest only performing the group sync to see what you get. The following properties should be adjusted:   <property name="Page Size">500</property> <property name="Sync Interval">30 mins</property> <property name="User Search Base"></property> <property name="User Object Class"></property> <property name="User Search Scope">SUBTREE</property> <property name="User Search Filter"></property> <property name="User Identity Attribute">sAMAccountName</property> <property name="User Group Name Attribute"></property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="Group Search Base">OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Group Object Class">group</property> <property name="Group Search Scope">SUBTREE</property> <property name="Group Search Filter">(|(cn=AG*)(cn=UG*))</property> <property name="Group Name Attribute">sAMAccountName</property> <property name="Group Member Attribute">member</property> <property name="Group Member Attribute - Referenced User Attribute"></property>   The above changes assume that your groups have "member" attribute. I recommend always setting a page size of either 500 to avoid missing returns for large queries. I do not recommend re-syncing users and groups every 2 mins as you had originally set.  This adds unnecessary load on CPU.  Keep in mind that all user and group identities synced are loaded into NiFi's heap memory. The above setup will return all groups starting from the group search base that start with yoru configured search filter.   From each returned group all the member lines will be returned which should contain the full DN for user members.  Those returned DNS are then looked up to return the "sAMAccountName" string for each user member.  I saw you were using full DNs for yoru groups previously (which is ok).    The sAMAccountName string for your groups will also be used as the group identity. You can also out the ldap-user-group-provider class in debug in the NiFi logback.xml to get output of the user and group identity strings synced with ease sync execution: org.apache.nifi.ldap.tenants.LdapUserGroupProvider   Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@bgumis  One way you can accomplish what you want to do is using an UpdateAttribute processor and RouteOnAttribute processor between your PutEmail and InvokeHTTP processors. The UpdateAttribute would be used to add an attribute to the FlowFile with the current time in milliseconds using NiFi Expression Language (NEL) statement: ${now():toNumber()}   The success relationship of the update Attribute processor would be routed to the RouteOnAttribute processor.   In the RouteOnAttribute processor you setup a new property that compares the records timestamp added to FlowFile by UpdateAttribute against the current time in milliseconds to see if 24 hours has passed yet. If true, the FlowFile will route to the expression property name relationship. If false, the FlowFile will route to the "unmatched" relationship which can be looped back on RouteOnAttribute.    The "unmatched" relationship should be configured to use "retry" to trigger penalization of FlowFiles.  This helps limit cpu usage by slowing how often the FlowFile is re-processed by penalizing the FlowFile on each retry.  The "Retry Max Back Off Period" would translate into the max time beyond 24 hours a FlowFile may get routed to the invokeHTTP.  I assume 24 hours is a min wait period and not a hard limit. The NEL statement used in RouteOnAttribute would look like this: ${now():toNumber():minus(${emailTime}):ge('86400000')}   The other option would require a custom script to set penalization of 24 hours on each FlowFile after the putEmail processor.   Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@sha257  This is typically caused by a bad configuration in the ldap-user-group-provider in the authorizers.xml. It would be difficult for me to provide and configuration specifics with out a sample output from your AD for a group and one of the group members. However, if you can share your ldap-user-group-provider configuration (minus any sensitive values), I may be able to offer some suggestions. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@rajivswe_2k7  Why are you fetching same files twice? I don't follow the "fail" all 5 if any one of them fails.  You successfully wrote some of them to destinations.  So what action are you taking when a partial failure happens (for example only 1 of 5 fails to write to archive? Why not just build dataflow around failure relationships to notify you of the specific files that failed? Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@phadkev  Would need to see more of the nifi-app.log to better understand what is going on here. Are you seeing the same org.apache.nifi.controller.StandardProcessorNode Timed out while waiting for OnScheduled exception for other components or just this executeScript processor? The exception itself is generic and could be thrown for any processor class. Are you ever seeing the log line telling you the NiFi UI is available at the following urls? If so NiFi is up. Are you seeing NiFi shut back down with some exception and stack trace in the nifi-app.log. What you shared implies NiFi is having issues scheduling this specific processor to execute.  This could very well be caused by an issue with the custom script that was build and used in this processor. If you NiFi is really not coming up, you could modify the nifi.properties file by changing  "nifi.flowcontroller.autoResumeState=true" to "nifi.flowcontroller.autoResumeState=false".  This will allow you NiFi to start without starting any processors.  You could then search the UI for the ExecuteScript component processor with id "acb441ba-c36b-1fdd-53f2-3a4821d43833".    Disable it and start all your other processors.  Restart your NiFi to see if you still have any issues.   This isolates the issue to this processor and your script. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more