@edaley  I also recommend upgrading to latest NiFi release to see if this issue persists.  I believe there was a known scheduling on startup bug in Apache NiFi 1.16 that since has been resolved. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@ajignacio  In the stack trace yoou'll see below: Caused by: org.apache.lucene.store.AlreadyClosedException: Underlying file changed by an external force   This indicates that something external to your NiFi is changing the provenance files.  When we see this it is most commonly the result of some virus software scanning the NiFi repositories and during that modifying the the files.    You should make sure that any external service exclude the NiFi repositories from scanning and modification. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@J_123  This is a 6 year old post.  NiFi Templates are close to point of deprecation from NiFi.  A NiFi is and has always been a snapshot of a specific set of processors.  It has no mechanism for tracking its reuse within the same NiFi where it was used.  Lets say someone creates a template, reuses it on the same NiFi multiple times, and then deletes the template from the NiFi (which is recommended since template like the rest of the flow on he canvas reside in NIFi's heap memory), how then do we track and replicate those changes.  Memory usage and this tracking are a few reasons why templates are in the way out and why we no longer recommend using this capability. Back at the end of 2021 (long after this post), NiFi introduced a new NiFi-Registry[1] service that is installed separate form NiFi and integrates with the NiFi service.  NiFi-Registry allows users to build dataflows on the a NiFi canvas and then version control each of those flows into NiFi-Registry.  Once version controlled into NiFi-Registry, that version controlled flow can be reuses on the same NiFi or any other NiFi using that same NiFi-Registry. If a user makes a change to anyone of these version flows linked to version control, NiFi will prompt that local changes exist allowing user to commit those locL changes to the copy in the NiFi-Registry.  Then all the other places (across multiple NiFi's if using that same dataflow from NiFi-Registry) where that version controlled flow is being used will prompt that a newer version is available allowing users to apply those changes.   If you have questions about something in NiFi and you cannot find a recent thread on the topic to raise a new question in the community to get the most up-to-date information. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt   [1] https://nifi.apache.org/docs/nifi-registry-docs/index.html ... View more

@myuintelli2021  Hello Ming, NiFi 1.15.3 will support JDK 1.8 or 1.11.  We do strongly encourage users to be on the latest update version of either of those with NiFI.  So not sure what update release of JDK your are on and which JDK provider (Oracle, OpenJDK, etc) you are using. I unfortunately do not have access ti a Windows 2019 datacenter edition to  see if I can reproduce myself to evaluate further. I strongly encourage you to raise a community question to take your query further.  A new question will gather more attention rather than trying to diagnosis and resolve your specific issue with the comments of a community article. Thank you, Matt ... View more

@steven-matison That 3 part series on ExecuteScript was written by a very talented different Matt in this community  @mburgess. ... View more

I was not able to successfully launch a basic cluster.  I closed this ticket because I did not see any followon.     ... View more

Hello, Thank you Matt for your quick response (and sorry for my late response). It work perfectly.   Best Regards   ... View more

@srilakshmi  Logging does not happen at the process group level.  Processors logging is based on the processor class.  So there is nothing in the log output produced by a processor within a process group that is going to tell you in which process group that particular processor belongs.  That being said, you may be able to prefix every processor's name within the same Process group with some string that identifies the process group.  This processor name would generally be included in the the log output produced by the processor.  Then you may be able to use logback filters (have not tried this myself) to filter log output based on these unique strings. https://logback.qos.ch/manual/filters.html   NiFi bulletins (bulletins are log output to the NiFi UI and have a rolling 5 minute life in the UI) however do include details about the parent Process Group in which the component generating the bulletin resides.   You could build a dataflow in yoru NiFi to handle bulletin notification through the use of the SiteToSiteBulletinReportingTask which is used to send bulletin to a destination remote import port on a target NiFi.  A dataflow on the target NiFi could be built to parse the received bulletin records by the bulletinGroupName json path property so that all records from same PG are kept together.  These  'like' records could then be written out to local filesystem based on the PG name, remote system, used to send email notifications, etc... Example of what a Bulletin sent using the SiteToSiteBulletinReportingTask looks like: { "objectId" : "541dbd22-aa4b-4a1a-ad58-5d9a0b730e42", "platform" : "nifi", "bulletinId" : 2200, "bulletinCategory" : "Log Message", "bulletinGroupId" : "7e7ad459-0185-1000-ffff-ffff9e0b1503", "bulletinGroupName" : "PG2-Bulletin", "bulletinGroupPath" : "NiFi Flow / Matt's PG / PG2-Bulletin", "bulletinLevel" : "DEBUG", "bulletinMessage" : "UpdateAttribute[id=8c5b3806-9c3a-155b-ba15-260075ce9a6f] Updated attributes for StandardFlowFileRecord[uuid=1b0cb23a-75d8-4493-ba82-c6ea5c7d1ce3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1672661850924-5, container=default, section=5], offset=969194, length=1024],offset=0,name=bulletin-${nextInt()).txt,size=1024]; transferring to 'success'", "bulletinNodeId" : "e75bf99f-095c-4672-be53-bb5510b3eb5c", "bulletinSourceId" : "8c5b3806-9c3a-155b-ba15-260075ce9a6f", "bulletinSourceName" : "PG1-UpdateAttribute", "bulletinSourceType" : "PROCESSOR", "bulletinTimestamp" : "2023-01-04T20:38:27.776Z" } In the above produced bulletin json you see the BulletinGroupName and the BulletinMessage (the actual log output).   If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more