I was not able to successfully launch a basic cluster.  I closed this ticket because I did not see any followon.     ... View more

Hello, Thank you Matt for your quick response (and sorry for my late response). It work perfectly.   Best Regards   ... View more

@srilakshmi  Logging does not happen at the process group level.  Processors logging is based on the processor class.  So there is nothing in the log output produced by a processor within a process group that is going to tell you in which process group that particular processor belongs.  That being said, you may be able to prefix every processor's name within the same Process group with some string that identifies the process group.  This processor name would generally be included in the the log output produced by the processor.  Then you may be able to use logback filters (have not tried this myself) to filter log output based on these unique strings. https://logback.qos.ch/manual/filters.html   NiFi bulletins (bulletins are log output to the NiFi UI and have a rolling 5 minute life in the UI) however do include details about the parent Process Group in which the component generating the bulletin resides.   You could build a dataflow in yoru NiFi to handle bulletin notification through the use of the SiteToSiteBulletinReportingTask which is used to send bulletin to a destination remote import port on a target NiFi.  A dataflow on the target NiFi could be built to parse the received bulletin records by the bulletinGroupName json path property so that all records from same PG are kept together.  These  'like' records could then be written out to local filesystem based on the PG name, remote system, used to send email notifications, etc... Example of what a Bulletin sent using the SiteToSiteBulletinReportingTask looks like: { "objectId" : "541dbd22-aa4b-4a1a-ad58-5d9a0b730e42", "platform" : "nifi", "bulletinId" : 2200, "bulletinCategory" : "Log Message", "bulletinGroupId" : "7e7ad459-0185-1000-ffff-ffff9e0b1503", "bulletinGroupName" : "PG2-Bulletin", "bulletinGroupPath" : "NiFi Flow / Matt's PG / PG2-Bulletin", "bulletinLevel" : "DEBUG", "bulletinMessage" : "UpdateAttribute[id=8c5b3806-9c3a-155b-ba15-260075ce9a6f] Updated attributes for StandardFlowFileRecord[uuid=1b0cb23a-75d8-4493-ba82-c6ea5c7d1ce3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1672661850924-5, container=default, section=5], offset=969194, length=1024],offset=0,name=bulletin-${nextInt()).txt,size=1024]; transferring to 'success'", "bulletinNodeId" : "e75bf99f-095c-4672-be53-bb5510b3eb5c", "bulletinSourceId" : "8c5b3806-9c3a-155b-ba15-260075ce9a6f", "bulletinSourceName" : "PG1-UpdateAttribute", "bulletinSourceType" : "PROCESSOR", "bulletinTimestamp" : "2023-01-04T20:38:27.776Z" } In the above produced bulletin json you see the BulletinGroupName and the BulletinMessage (the actual log output).   If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@RodolfoE, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  ... View more

Thanks Matt,I will test and Let you know ... View more

Hi @MattWho , After i Enabled Retrieve site-to-site details for the Input port BulleInfo, then i was able to receive the BulletinInfo.   Thank you so much for you suggestions!!   Regards, Pradeep ... View more

@samrathal  1. What is the purpose of the SplitJson in your dataflow? 2. If you have 1 FlowFile with 1000 records in it, why use SplitJson to split that in to 1000 FlowFiles having 1 record each?  Why not just merge the larger FlowFiles with multiple records in it?  Or am i missing part of the use case here? --- Can you share a template of flow definition of yoru dataflow? 1. It is not clear to me how you get "X-Total-Count" and how you are adding this FlowFile attribute to every FlowFile. 2. You have configured the "Release Signal Identifier" with a boolean NiFi Expression Language (NEL) that using your example will return "false" until "fragment.count" FlowFile attribute value equals the FlowFile attribute "X-Total-Count" value.          2a. I assume you are writing "X-Total-Count" to every FlowFile coming out of the SplitJson?  How are incrementing the "fragment.count" across all FlowFile in the complete 5600 record batch.  Each FlowFile that splits into 1000 FlowFiles via splitJson will have fragment.count set to 1 - 1000.  So fragment.count would never reach 5600 unless you are handling this count somewhere else in your dataflow.         2b. If a FlowFile where value from "fragment.count" actually equals value from "X-Total-Count" attribute, your "Release Signal Identifier" will resolve to "true".  The ""Release Signal Identifier" value (true or false) in your configuration is looked up in the configured "distributed map cache server.  So where in your dataflow to you write the release signal to the distributed map cache? (usually handled by a notify processor) I am in no way implying that what you are trying to accomplish can't be done.  However, coming up with an end-to-end workable solution requires knowing all the steps in the use case along the way.   I would recommend going through the example Wait/Notify linked in my original response to get a better understanding of how wait and notify processors work together.  Then maybe you can makes some changes to your existing dataflow implementation.  With more use case details (detailed process steps) I could suggest further changes if needed. I really hope this helps you get some traction on your use case here. If you have a contract with Cloudera, you can reach out to your  account owner who could help arrange for professional services that can work with your to solution your use cases in to workable NiFi dataflows. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more